1989issue C011-3
Binary search limits on multi-peak average grids
A worked system-optimization example mapped two exponential-average decimals on one historical database. The grid formed several peaks and a valley, so a binary search from one start could miss another high. Even a sound search still left a robustness problem, because later data seldom match the sample used to set the pair.
- A worked example tuned only two moving-average inputs, one exponential-average decimal for buying and one for selling, and applied the same historical database to every pair.
- The parameter-step-grid formed several peaks, a valley along one sell-decimal line, and local dips rather than a single hill.
- Binary optimization with an initial increment of 0.05 can miss other peaks; four illustrated starting pairs each failed to reach at least one other observed peak.
- Binary optimization was treated as appropriate only on a single-peak surface, and choosing rules on historical prices remains a robustness problem because later data seldom match the sample used to set the parameters.
Two decimals on one database
A worked system-optimization example tuned only two moving-average inputs: one exponential-average decimal for buying and one for selling.
The buy decimal was stepped from 0.1 to 0.3 by 0.05 and the sell decimal from 0.15 to 0.35 by 0.05, with the same historical database applied to every pair. That listing is a parameter-step-grid.
Several peaks and a valley
Results on that two-parameter moving-average grid formed several peaks, a valley along one sell-decimal line, and local dips rather than a single hill that rose and fell in one direction.
The map is a multi-peak-surface: more than one local high, plus valleys or dips, so a climb from one start need not reach another high.
Varying more than two parameters together was expected to make the optimization surface still more complex.
The binary path depends on the start
Binary optimization is a successive-step search that moves from a chosen starting pair and increment rather than scoring every cell on a parameter grid.
A binary search with an initial increment of 0.05 can miss other peaks. The path depends on that increment and on the starting buy and sell values.
Four illustrated starting pairs were (0.10, 0.15), (0.15, 0.35), (0.30, 0.30), and (0.30, 0.20). Each was described as unable to reach at least one of the other observed peaks.
Binary optimization was said to reduce computing time but to be appropriate only when the objective surface has a single peak.
The sample used to set the pair
Even if a search method is sound, choosing rules on historical prices remains a robustness problem because later data seldom match the sample used to set the parameters.
A contemporaneous test that found no benefit from optimization was faulted for not stating the search procedure or whether the testers faced a single-peak surface.
All readings on this track · 51 readings
- 1986Degrees of freedom in trading system optimization
- 1988Walk-forward and neighborhood tests after optimization
- 1988Undisclosed rules block system robustness tests
- 1988Testing re-optimization calendars against random parameter controls
- 1989Binary search limits on multi-peak average grids
- 1989Parameter neighborhoods that survive a shift
- 1990Use profit mapping to keep a cycle and stop plateau
- 1990Why popular indicator optimization fails robustness
- 1991Retesting weighted indicator balances across horizons
- 1992Constructing forecast models with regression, walk-forward, and robustness
- 1992Diagnose regimes before you lock parameters
- 1992When stops change system timing
- 1993Walk-forward halt rules for forecast models
- 1994Walk-forward evaluation of genetic index rules
- 1995Input pruning as walk-forward system evaluation
- 1995Critiquing neural nets as incomplete trading systems
- 1996Rebuild the equity-path ratio before it ranks a designed system
- 1996Parameter grids can fit random walks
- 1996Walk-forward analysis belongs in the design of a mechanical trading system
- 1997When a holdout fails, discard the rule set
- 1997Test rewarded rule breaks before replacing the system
- 1997Walk-forward rules keep system research from rewriting live trades
- 1999Keep a channel-breakout to two lookbacks and test neighbor stability
- 1999Constant investment size in stock system evaluation
- 2000Forcing optimization maps mechanical system failure boundaries
- 2000Robust parameter selection with surface charts
- 2001A two-gate classroom test for a two-window momentum trend filter
- 2002How a two-sided continuation factor becomes a testable trend rule
- 2002Evaluating two-window trend intensity as a reversal rule
- 2003Discounting speculative bubbles in system robustness tests
- 2003Walk-forward evaluation of locked stochastic oscillator rules
- 2003Critiquing mechanical system design after extreme price regimes
- 2004Evaluating a two-window trend trigger
- 2005Grade backtested signals with holdouts and optimization plateaus
- 2006Reserved-sample evaluation of trading system design
- 2006Walk-forward critique of hindsight crossover systems
- 2008Condition-matched walk-forward evaluation for mechanical systems
- 2011Session-split evaluation of regular and overnight systems
- 2012Walk-forward evaluation as operator rehearsal
- 2013Two-window evaluation of mechanical trading systems
- 2013Walk-forward filter selection for repeated-median velocity
- 2014Walk-forward evaluation for fading-memory velocity systems
- 2015Test oscillator events before tuning rules
- 2016Walk-forward evaluation of a five-parameter parabolic stop-and-reversal
- 2016Walk-forward optimization without curve fitting
- 2017Optimization without overfitting in trend-system evaluation
- 2017Parameter stability is a better guide than a larger crossover grid
- 2018Point-in-time universes for system evaluation
- 2018Walk-forward robustness evaluation for optimized systems
- 2018Critiquing breakout systems through robustness tests
- 2018A critique of parameter fitting in system design