Skip to main content
Track Robustness testing
7 / 51
Library

1990issue C041-7

Use profit mapping to keep a cycle and stop plateau

A two-parameter profit map is a robustness test of dominant-cycle length and initial stop offset, not a search for the historically richest pair. Keep the pair that sits on a parameter plateau and set aside a taller peak when it sits on a steep drop.

  • Searching every combination of two rule parameters for the single historically highest profit is treated as fitting the model to back data and is not, by itself, evidence that those parameters will remain useful after market characteristics change.
  • Profit mapping plots profit as height over two independent rule parameters so a designer can find a parameter plateau where small moves in both inputs leave results nearly unchanged.
  • Low-sensitivity regions are often distant from the historically richest pair. For a cycle-timed entry and an accelerating stop, the mapped axes are dominant-cycle length and initial stop offset.
  • When signal-to-noise ratio is poor and parameter sensitivity is high, the mapping procedure treats the surface as a reason not to trade rather than a reason to take the peak.
Entries in this reading3 entries

The historically richest pair is not the test

Searching every combination of two rule parameters for the single historically highest profit is treated as fitting the model to back data. That search is not, by itself, evidence that those parameters will remain useful after market characteristics change.

In this workflow that exhaustive search is system optimization. It is treated as curve-fitting unless it is replaced by a low-sensitivity region.

Read the map by slope as well as height

Profit mapping is a grid of historical system results over two independent rule parameters, plotted so profit is height and each pair can be judged by slope as well as height. A three-dimensional map with one parameter on each horizontal axis and profit as height is used to locate a region where small moves in both parameters leave profit nearly unchanged.

Parameter sensitivity is how sharply mapped profit changes when either rule parameter is moved a small step away from a candidate pair. A parameter plateau is a region of the profit surface where results stay acceptable while both parameters vary, and that region is preferred over an isolated peak.

Low-sensitivity regions on that map are often distant from the parameter pair that produced the single highest historical profit. Robustness testing judges a parameter pair by whether profit stays stable under parameter steps, added noise, and a longer sample window.

Systems that already have two parameters

Two-average crossovers, triple averages with two legs held in a fixed ratio, stochastics, relative-strength index, and parabolic stop-and-reverse are presented as systems that already have two parameters or can be rewritten so the same mapping applies.

Axes for a cycle-timed entry and an accelerating stop

For a cycle-timed entry paired with an accelerating stop, the mapped axes are dominant-cycle length and initial stop offset. The dominant cycle is the measured periodic length used as one independent input to a cycle-timed entry and as the lookback that scales the initial stop.

Initial stop offset is a multiplier, allowed to range from 0.25 to 2.5, applied to average daily range over the most recent half cycle to set the first stop distance from the entry-day price extreme.

A clean theoretical cycle formed a ridge

On a noise-free theoretical 20-day cycle spanning 80 days, profit formed a ridge centered on a 20-day dominant cycle and showed no material sensitivity to the offset parameter. Overestimating cycle length reduced profit less than underestimating it.

Noise can turn a peak into a cliff

Noise at one-fourth the sine-wave amplitude left the profitable parameter region largely unchanged. Noise equal to the sine-wave amplitude kept the peak at a 20-day cycle only at the largest offset, and that peak sat beside a steep drop into losses.

Signal-to-noise ratio is the relative strength of the cyclic component versus additive noise. When parameter sensitivity is high because the signal-to-noise ratio is poor, the mapping procedure treats the surface as a reason not to trade rather than a reason to take the peak. When that ratio is poor the surface becomes too peaked to justify selecting any pair.

A longer window is used to check the plateau

A two-month pork-bellies map placed the historically richest pair at a 32-day dominant cycle and a 0.25 offset on a steep drop, while a 12-day cycle with a 0.5 offset sat in the center of a broader mound. A full-year map of the same contract kept that two-month plateau pair inside a still-consistent profitable-parameter region.

Educational research material, not investment advice. Historical source context does not establish present-day performance.
7 of 51 in the Robustness testing track
19901-2 pp.Next on Robustness testingWhy popular indicator optimization fails robustnessA limited public catalog of popular indicators is already thoroughly examined, so further remixes of those same inputs are unlikely to be undiscovered.
All readings on this track · 51 readings
  1. 1986Degrees of freedom in trading system optimization
  2. 1988Walk-forward and neighborhood tests after optimization
  3. 1988Undisclosed rules block system robustness tests
  4. 1988Testing re-optimization calendars against random parameter controls
  5. 1989Binary search limits on multi-peak average grids
  6. 1989Parameter neighborhoods that survive a shift
  7. 1990Use profit mapping to keep a cycle and stop plateau
  8. 1990Why popular indicator optimization fails robustness
  9. 1991Retesting weighted indicator balances across horizons
  10. 1992Constructing forecast models with regression, walk-forward, and robustness
  11. 1992Diagnose regimes before you lock parameters
  12. 1992When stops change system timing
  13. 1993Walk-forward halt rules for forecast models
  14. 1994Walk-forward evaluation of genetic index rules
  15. 1995Input pruning as walk-forward system evaluation
  16. 1995Critiquing neural nets as incomplete trading systems
  17. 1996Rebuild the equity-path ratio before it ranks a designed system
  18. 1996Parameter grids can fit random walks
  19. 1996Walk-forward analysis belongs in the design of a mechanical trading system
  20. 1997When a holdout fails, discard the rule set
  21. 1997Test rewarded rule breaks before replacing the system
  22. 1997Walk-forward rules keep system research from rewriting live trades
  23. 1999Keep a channel-breakout to two lookbacks and test neighbor stability
  24. 1999Constant investment size in stock system evaluation
  25. 2000Forcing optimization maps mechanical system failure boundaries
  26. 2000Robust parameter selection with surface charts
  27. 2001A two-gate classroom test for a two-window momentum trend filter
  28. 2002How a two-sided continuation factor becomes a testable trend rule
  29. 2002Evaluating two-window trend intensity as a reversal rule
  30. 2003Discounting speculative bubbles in system robustness tests
  31. 2003Walk-forward evaluation of locked stochastic oscillator rules
  32. 2003Critiquing mechanical system design after extreme price regimes
  33. 2004Evaluating a two-window trend trigger
  34. 2005Grade backtested signals with holdouts and optimization plateaus
  35. 2006Reserved-sample evaluation of trading system design
  36. 2006Walk-forward critique of hindsight crossover systems
  37. 2008Condition-matched walk-forward evaluation for mechanical systems
  38. 2011Session-split evaluation of regular and overnight systems
  39. 2012Walk-forward evaluation as operator rehearsal
  40. 2013Two-window evaluation of mechanical trading systems
  41. 2013Walk-forward filter selection for repeated-median velocity
  42. 2014Walk-forward evaluation for fading-memory velocity systems
  43. 2015Test oscillator events before tuning rules
  44. 2016Walk-forward evaluation of a five-parameter parabolic stop-and-reversal
  45. 2016Walk-forward optimization without curve fitting
  46. 2017Optimization without overfitting in trend-system evaluation
  47. 2017Parameter stability is a better guide than a larger crossover grid
  48. 2018Point-in-time universes for system evaluation
  49. 2018Walk-forward robustness evaluation for optimized systems
  50. 2018Critiquing breakout systems through robustness tests
  51. 2018A critique of parameter fitting in system design
All 58 readings tagged Robustness testing
Also on Robustness testing5 readings