Skip to main content
Track Robustness testing
22 / 51
Search the archiveLibrary
Library

1997issue C081-6

Walk-forward rules keep system research from rewriting live trades

A mechanical trading system is a written long, short, or flat stance that live execution must match. The archive workflow froze that book for six months, kept research off-book, and judged robustness testing across a five-year span rather than a short live window.

  • A mechanical trading system is complete only when written rules name the long, short, or flat stance at every instant and live execution matches that stance.
  • Discretionary oscillator reading was abandoned for a complete rule procedure, and even a coded always-in RSI book was withdrawn from live use after a five-year historical run.
  • Robustness testing looks for a wide plateau of parameter sets that move drawdown, profit factor, and average trade as trade frequency changes, not for a single net-profit peak.
  • A six-month freeze keeps research off-book and forbids live edits until the scheduled review; passing trades or resizing from opinion breaks the match to the hypothetical path.
Entries in this reading3 entries
Current trackRobustness testing58 readingsWalk-forward analysis95 readingsMechanical trading system158 readings

A written stance at every instant

A mechanical trading system is defined by knowing the required long, short, or flat stance at every instant from written rules, then matching that hypothetical stance in live execution.

Indicator construction knowledge was treated as insufficient. Discretionary oscillator reading produced many daily trades and was abandoned in favor of a complete rule procedure.

June 1997 S&P futures while Always-in RSI stayed long or short

The June 1997 S&P contract climbed from the high 760s to the high 830s between 22 April and 14 May while Saidenberg’s Always-in RSI book flipped long and short on the marked turns. Session levels were read from the printed chart; the interview did not publish a price table.
The June 1997 S&P contract climbed from the high 760s to the high 830s between 22 April and 14 May while Saidenberg’s Always-in RSI book flipped long and short on the marked turns. Session levels were read from the printed chart; the interview did not publish a price table.June 1997 S&P futures · 30-minute bars · 1997-04-22T00:00:00.000Z to 1997-05-14T00:00:00.000Z

Prices are approximate to the nearest index point. The magazine chart is a 30-minute Always-in RSI plot; individual bar closes cannot be resolved, so each labeled session is one digitized point. He later dropped the rules after a five-year test showed drawdowns near $50,000.

Code the procedure, then withdraw it

One early always-in procedure used a nine-period RSI on 30-minute bars, double-bottom and double-top criteria around the 30 and 70 lines, and a new-low reverse so a failed long could flip short.

After that RSI procedure was coded and run over a five-year history, it was withdrawn from live use rather than treated as finished.

A plateau instead of a peak

Robustness testing is judged by a wide plateau of parameter sets that move drawdown, profit factor, and average trade in the expected direction when trade frequency changes, not by a single net-profit peak.

A six-month test window, even with 200 or 300 trades, was treated as too short because another six-month slice inside a five-year span can look different. Intraday robustness work used five years of actual-contract tests, then joined three-month segments into one equity path instead of relying on a single built-in continuous series.

Research continues, the live book does not

A lifetime pledge to one book is rejected because the operator will abandon it at the worst moment. A six-month freeze lets research continue while forbidding live edits until the scheduled review.

After each six-month window the book may add, drop, or modify systems so the next period again has a known daily procedure.

Opinion breaks the hypothetical path

Passing trades or resizing from opinion converts a mechanical book back into discretion and breaks the match to the hypothetical path.

Educational research material, not investment advice. Historical source context does not establish present-day performance.
22 of 51 in the Robustness testing track
19991-10 pp.Next on Robustness testingKeep a channel-breakout to two lookbacks and test neighbor stabilityA channel-breakout changes stance when price penetrates the highest high or lowest low of a lookback-window, and only one long-entry length and one long-exit length are treated as the critical knobs.
All readings on this track · 51 readings
  1. 1986Degrees of freedom in trading system optimization
  2. 1988Walk-forward and neighborhood tests after optimization
  3. 1988Undisclosed rules block system robustness tests
  4. 1988Testing re-optimization calendars against random parameter controls
  5. 1989Binary search limits on multi-peak average grids
  6. 1989Parameter neighborhoods that survive a shift
  7. 1990Use profit mapping to keep a cycle and stop plateau
  8. 1990Why popular indicator optimization fails robustness
  9. 1991Retesting weighted indicator balances across horizons
  10. 1992Constructing forecast models with regression, walk-forward, and robustness
  11. 1992Diagnose regimes before you lock parameters
  12. 1992When stops change system timing
  13. 1993Walk-forward halt rules for forecast models
  14. 1994Walk-forward evaluation of genetic index rules
  15. 1995Input pruning as walk-forward system evaluation
  16. 1995Critiquing neural nets as incomplete trading systems
  17. 1996Rebuild the equity-path ratio before it ranks a designed system
  18. 1996Parameter grids can fit random walks
  19. 1996Walk-forward analysis belongs in the design of a mechanical trading system
  20. 1997When a holdout fails, discard the rule set
  21. 1997Test rewarded rule breaks before replacing the system
  22. 1997Walk-forward rules keep system research from rewriting live trades
  23. 1999Keep a channel-breakout to two lookbacks and test neighbor stability
  24. 1999Constant investment size in stock system evaluation
  25. 2000Forcing optimization maps mechanical system failure boundaries
  26. 2000Robust parameter selection with surface charts
  27. 2001A two-gate classroom test for a two-window momentum trend filter
  28. 2002How a two-sided continuation factor becomes a testable trend rule
  29. 2002Evaluating two-window trend intensity as a reversal rule
  30. 2003Discounting speculative bubbles in system robustness tests
  31. 2003Walk-forward evaluation of locked stochastic oscillator rules
  32. 2003Critiquing mechanical system design after extreme price regimes
  33. 2004Evaluating a two-window trend trigger
  34. 2005Grade backtested signals with holdouts and optimization plateaus
  35. 2006Reserved-sample evaluation of trading system design
  36. 2006Walk-forward critique of hindsight crossover systems
  37. 2008Condition-matched walk-forward evaluation for mechanical systems
  38. 2011Session-split evaluation of regular and overnight systems
  39. 2012Walk-forward evaluation as operator rehearsal
  40. 2013Two-window evaluation of mechanical trading systems
  41. 2013Walk-forward filter selection for repeated-median velocity
  42. 2014Walk-forward evaluation for fading-memory velocity systems
  43. 2015Test oscillator events before tuning rules
  44. 2016Walk-forward evaluation of a five-parameter parabolic stop-and-reversal
  45. 2016Walk-forward optimization without curve fitting
  46. 2017Optimization without overfitting in trend-system evaluation
  47. 2017Parameter stability is a better guide than a larger crossover grid
  48. 2018Point-in-time universes for system evaluation
  49. 2018Walk-forward robustness evaluation for optimized systems
  50. 2018Critiquing breakout systems through robustness tests
  51. 2018A critique of parameter fitting in system design
By fieldSystem design117 readingsBy year199756 readings
All 58 readings tagged Robustness testing
Also on Robustness testing5 readings
1996Walk-forward analysis belongs in the design of a mechanical trading system2000Forcing optimization maps mechanical system failure boundaries2008Condition-matched walk-forward evaluation for mechanical systems2013Two-window evaluation of mechanical trading systems1986Degrees of freedom in trading system optimization