1990issue C041-7
Use profit mapping to keep a cycle and stop plateau
A two-parameter profit map is a robustness test of dominant-cycle length and initial stop offset, not a search for the historically richest pair. Keep the pair that sits on a parameter plateau and set aside a taller peak when it sits on a steep drop.
- Searching every combination of two rule parameters for the single historically highest profit is treated as fitting the model to back data and is not, by itself, evidence that those parameters will remain useful after market characteristics change.
- Profit mapping plots profit as height over two independent rule parameters so a designer can find a parameter plateau where small moves in both inputs leave results nearly unchanged.
- Low-sensitivity regions are often distant from the historically richest pair. For a cycle-timed entry and an accelerating stop, the mapped axes are dominant-cycle length and initial stop offset.
- When signal-to-noise ratio is poor and parameter sensitivity is high, the mapping procedure treats the surface as a reason not to trade rather than a reason to take the peak.
The historically richest pair is not the test
Searching every combination of two rule parameters for the single historically highest profit is treated as fitting the model to back data. That search is not, by itself, evidence that those parameters will remain useful after market characteristics change.
In this workflow that exhaustive search is system optimization. It is treated as curve-fitting unless it is replaced by a low-sensitivity region.
Read the map by slope as well as height
Profit mapping is a grid of historical system results over two independent rule parameters, plotted so profit is height and each pair can be judged by slope as well as height. A three-dimensional map with one parameter on each horizontal axis and profit as height is used to locate a region where small moves in both parameters leave profit nearly unchanged.
Parameter sensitivity is how sharply mapped profit changes when either rule parameter is moved a small step away from a candidate pair. A parameter plateau is a region of the profit surface where results stay acceptable while both parameters vary, and that region is preferred over an isolated peak.
Low-sensitivity regions on that map are often distant from the parameter pair that produced the single highest historical profit. Robustness testing judges a parameter pair by whether profit stays stable under parameter steps, added noise, and a longer sample window.
Systems that already have two parameters
Two-average crossovers, triple averages with two legs held in a fixed ratio, stochastics, relative-strength index, and parabolic stop-and-reverse are presented as systems that already have two parameters or can be rewritten so the same mapping applies.
Axes for a cycle-timed entry and an accelerating stop
For a cycle-timed entry paired with an accelerating stop, the mapped axes are dominant-cycle length and initial stop offset. The dominant cycle is the measured periodic length used as one independent input to a cycle-timed entry and as the lookback that scales the initial stop.
Initial stop offset is a multiplier, allowed to range from 0.25 to 2.5, applied to average daily range over the most recent half cycle to set the first stop distance from the entry-day price extreme.
A clean theoretical cycle formed a ridge
On a noise-free theoretical 20-day cycle spanning 80 days, profit formed a ridge centered on a 20-day dominant cycle and showed no material sensitivity to the offset parameter. Overestimating cycle length reduced profit less than underestimating it.
Noise can turn a peak into a cliff
Noise at one-fourth the sine-wave amplitude left the profitable parameter region largely unchanged. Noise equal to the sine-wave amplitude kept the peak at a 20-day cycle only at the largest offset, and that peak sat beside a steep drop into losses.
Signal-to-noise ratio is the relative strength of the cyclic component versus additive noise. When parameter sensitivity is high because the signal-to-noise ratio is poor, the mapping procedure treats the surface as a reason not to trade rather than a reason to take the peak. When that ratio is poor the surface becomes too peaked to justify selecting any pair.
A longer window is used to check the plateau
A two-month pork-bellies map placed the historically richest pair at a 32-day dominant cycle and a 0.25 offset on a steep drop, while a 12-day cycle with a 0.5 offset sat in the center of a broader mound. A full-year map of the same contract kept that two-month plateau pair inside a still-consistent profitable-parameter region.
All readings on this track · 51 readings
- 1986Degrees of freedom in trading system optimization
- 1988Walk-forward and neighborhood tests after optimization
- 1988Undisclosed rules block system robustness tests
- 1988Testing re-optimization calendars against random parameter controls
- 1989Binary search limits on multi-peak average grids
- 1989Parameter neighborhoods that survive a shift
- 1990Use profit mapping to keep a cycle and stop plateau
- 1990Why popular indicator optimization fails robustness
- 1991Retesting weighted indicator balances across horizons
- 1992Constructing forecast models with regression, walk-forward, and robustness
- 1992Diagnose regimes before you lock parameters
- 1992When stops change system timing
- 1993Walk-forward halt rules for forecast models
- 1994Walk-forward evaluation of genetic index rules
- 1995Input pruning as walk-forward system evaluation
- 1995Critiquing neural nets as incomplete trading systems
- 1996Rebuild the equity-path ratio before it ranks a designed system
- 1996Parameter grids can fit random walks
- 1996Walk-forward analysis belongs in the design of a mechanical trading system
- 1997When a holdout fails, discard the rule set
- 1997Test rewarded rule breaks before replacing the system
- 1997Walk-forward rules keep system research from rewriting live trades
- 1999Keep a channel-breakout to two lookbacks and test neighbor stability
- 1999Constant investment size in stock system evaluation
- 2000Forcing optimization maps mechanical system failure boundaries
- 2000Robust parameter selection with surface charts
- 2001A two-gate classroom test for a two-window momentum trend filter
- 2002How a two-sided continuation factor becomes a testable trend rule
- 2002Evaluating two-window trend intensity as a reversal rule
- 2003Discounting speculative bubbles in system robustness tests
- 2003Walk-forward evaluation of locked stochastic oscillator rules
- 2003Critiquing mechanical system design after extreme price regimes
- 2004Evaluating a two-window trend trigger
- 2005Grade backtested signals with holdouts and optimization plateaus
- 2006Reserved-sample evaluation of trading system design
- 2006Walk-forward critique of hindsight crossover systems
- 2008Condition-matched walk-forward evaluation for mechanical systems
- 2011Session-split evaluation of regular and overnight systems
- 2012Walk-forward evaluation as operator rehearsal
- 2013Two-window evaluation of mechanical trading systems
- 2013Walk-forward filter selection for repeated-median velocity
- 2014Walk-forward evaluation for fading-memory velocity systems
- 2015Test oscillator events before tuning rules
- 2016Walk-forward evaluation of a five-parameter parabolic stop-and-reversal
- 2016Walk-forward optimization without curve fitting
- 2017Optimization without overfitting in trend-system evaluation
- 2017Parameter stability is a better guide than a larger crossover grid
- 2018Point-in-time universes for system evaluation
- 2018Walk-forward robustness evaluation for optimized systems
- 2018Critiquing breakout systems through robustness tests
- 2018A critique of parameter fitting in system design