20251-45
Add a second procedure before you retune the first
Once a first profitable procedure exists, the next design move is a second distinct entry-exit-abstention stack, not another search of the first. Editorial view: treat the live rule set as perishable, then refuse range tweaks chosen only because a full-history test improved.
- Once a first profitable procedure exists, field a second distinct entry-exit-abstention stack rather than keep refining the original one.
- A single successful rule set becomes part of trader identity and delays repair or abandonment after the edge decays slowly, without an obvious alert.
- Widening a lookback range only after that change improves a full-history test is an optimization of the range, not a harmless robustness tweak.
- After repeated backtesting and optimization, running fewer optimization passes is the more durable long-run design habit, even when a prettier historical equity path is discarded.
The next job after a working procedure
Trading-systems system design is the craft of specifying entry, exit and abstention as one testable procedure rather than a collection of isolated signals. Once a first profitable procedure exists, the highest-leverage design move is to field a second distinct entry-exit-abstention stack rather than to keep refining the original one.
Editorial view: a live rule set is a perishable procedure. The first design job is not another pass of system optimization on the same stack. It is a second independent stack, so identity and drawdown risk are no longer concentrated in one backtested edge.
Identity lock-in and silent decay
A single successful rule set becomes part of trader identity, which delays major repair or abandonment after the edge decays slowly and without an obvious alert.
Editorial view: when the procedure is also the trader's story about skill, slow deterioration does not force a decision. A second live stack reduces emotional ownership of any one rule set and makes repair or retirement easier to notice.
What a second stack changes
Two truly distinct procedures make combined equity more resilient to drawdowns than either stack alone and reduce emotional ownership of any one rule set. Running a second live procedure also improves later screening of ideas and the ability to notice differences between a backtest or paper run and live execution.
Editorial view: the second stack is not a spare copy of the first. It has to be a distinct entry-exit-abstention procedure, or identity and drawdown risk remain concentrated in one edge.
When a range tweak is another search
Widening a lookback range only after that change improves a full-history test is itself an optimization of the range, not a harmless robustness tweak. Robustness testing asks whether a procedure still deserves capital after small, identity-preserving changes, instead of after a search that was only run because results improved.
Re-optimizing an existing system solely because the new backtest looks better selects for curve-fit ranges; the opposite result would have been discarded. System optimization searches rule inputs against historical market state and execution constraints to improve a backtested signal. Editorial view: using that search again only because the historical path looks prettier converts a robustness check into an extra optimization pass.
How to keep a range comparison honest
A range comparison is acceptable only if the choice is made on a small data slice and a single winner is then evaluated once on the remaining history. Walk-forward analysis holds a candidate procedure to data that was not used to choose it, so the same entry-exit-abstention stack can be judged as a live process.
After repeated backtesting and optimization, running fewer optimization passes is the more durable long-run design habit, even when discarding a prettier historical equity path feels costly. Editorial view: refuse post-hoc range tweaks. The durable habit is fewer searches, not a prettier full-history path.
All readings on this track · 57 readings
- 1986Degrees of freedom in trading system optimization
- 1988Walk-forward and neighborhood tests after optimization
- 1988Undisclosed rules block system robustness tests
- 1988Testing re-optimization calendars against random parameter controls
- 1989Binary search limits on multi-peak average grids
- 1989Parameter neighborhoods that survive a shift
- 1990Use profit mapping to keep a cycle and stop plateau
- 1990Why popular indicator optimization fails robustness
- 1991Retesting weighted indicator balances across horizons
- 1992Constructing forecast models with regression, walk-forward, and robustness
- 1992Diagnose regimes before you lock parameters
- 1992When stops change system timing
- 1993Walk-forward halt rules for forecast models
- 1994Walk-forward evaluation of genetic index rules
- 1995Input pruning as walk-forward system evaluation
- 1995Critiquing neural nets as incomplete trading systems
- 1996Rebuild the equity-path ratio before it ranks a designed system
- 1996Parameter grids can fit random walks
- 1996Walk-forward analysis belongs in the design of a mechanical trading system
- 1997When a holdout fails, discard the rule set
- 1997Test rewarded rule breaks before replacing the system
- 1997Walk-forward rules keep system research from rewriting live trades
- 1999Keep a channel-breakout to two lookbacks and test neighbor stability
- 1999Constant investment size in stock system evaluation
- 2000Forcing optimization maps mechanical system failure boundaries
- 2000Robust parameter selection with surface charts
- 2001A two-gate classroom test for a two-window momentum trend filter
- 2002How a two-sided continuation factor becomes a testable trend rule
- 2002Evaluating two-window trend intensity as a reversal rule
- 2003Discounting speculative bubbles in system robustness tests
- 2003Walk-forward evaluation of locked stochastic oscillator rules
- 2003Critiquing mechanical system design after extreme price regimes
- 2004Evaluating a two-window trend trigger
- 2005Grade backtested signals with holdouts and optimization plateaus
- 2006Reserved-sample evaluation of trading system design
- 2006Walk-forward critique of hindsight crossover systems
- 2008Condition-matched walk-forward evaluation for mechanical systems
- 2011Session-split evaluation of regular and overnight systems
- 2012Walk-forward evaluation as operator rehearsal
- 2013Two-window evaluation of mechanical trading systems
- 2013Walk-forward filter selection for repeated-median velocity
- 2014Walk-forward evaluation for fading-memory velocity systems
- 2015Test oscillator events before tuning rules
- 2016Walk-forward evaluation of a five-parameter parabolic stop-and-reversal
- 2016Walk-forward optimization without curve fitting
- 2017Optimization without overfitting in trend-system evaluation
- 2017Parameter stability is a better guide than a larger crossover grid
- 2018Point-in-time universes for system evaluation
- 2018Walk-forward robustness evaluation for optimized systems
- 2018Critiquing breakout systems through robustness tests
- 2018A critique of parameter fitting in system design
- 2019Noise-matched rules still need trend filters and robustness tests
- 2019Three gates for evaluating a trading system
- 2020Data construction as a mechanical system input
- 2020Hidden optimization in ported relative-strength systems
- 2020When mechanical historical tests decay after optimization
- 2025Add a second procedure before you retune the first