Skip to main content
Track Robustness testing
40 / 51
Search the archiveLibrary
Library

2013issue C1038-41

Two-window evaluation of mechanical trading systems

A mechanical rule set is first viewed across about one thousand daily bars spanning more than one market type, then examined on the most recent three months. Equity curves are reviewed before the procedure is treated as ready to use.

  • A mechanical trading procedure is described as having both strengths and weaknesses, and no single system is presented as suitable for every trader or every market state.
  • Robustness is framed as repeating one rule set through bullish, bearish, and sideways stretches, where losing trades in a mismatch such as trend-following in a choppy market are treated as expected.
  • After a multi-year first window of about one thousand daily bars, the same entry, exit, and sit-out rules are examined on the most recent three months so recent signals are judged separately.
  • Reviewing equity curves is presented as a check to complete before treating a mechanical procedure as ready to use.
Entries in this reading3 entries
Current trackRobustness testing58 readingsWalk-forward analysis95 readingsMechanical trading system158 readings

A fully specified rule set

A mechanical trading system is a fully specified procedure that turns rule inputs, market state, and execution constraints into entry, exit, or sit-out signals over the system holding period. The whole rule set can then be tested as one process rather than as isolated chart opinions.

A mechanical trading procedure is described as having both strengths and weaknesses, with no single system presented as suitable for every trader or every market state.

Stress the same rules across market types

A first historical evaluation is recommended to use four years of data, described as about one thousand daily bars, so the same rules can be viewed across more than one market type.

Robustness testing repeats one rule set through bullish, bearish, and sideways stretches to locate where the procedure fits the market state and where it is mismatched. Robustness is framed as that multi-state check rather than a test confined to a single market phase.

When a trend-following procedure meets a choppy market, losing trades are treated as an expected mismatch between the rules and the current market state.

XLF weekly price with 10-week and 40-week moving averages, 2011–2012

Weekly XLF from December 2010 through 16 November 2012 shows an early-2011 rise to 16.320, the autumn 2011 washout to 10.749, and the 2012 rebound toward 16.390, so one ticker spans advance, crash, and recovery. Printed swing tags and the session last of 15.25 were read from the StockCharts figure; the 10-week and 40-week averages are the overlays on that same figure.
Weekly XLF from December 2010 through 16 November 2012 shows an early-2011 rise to 16.320, the autumn 2011 washout to 10.749, and the 2012 rebound toward 16.390, so one ticker spans advance, crash, and recovery. Printed swing tags and the session last of 15.25 were read from the StockCharts figure; the 10-week and 40-week averages are the overlays on that same figure.XLF · Weekly · 2010-12-01T00:00:00.000Z to 2012-11-30T00:00:00.000Z

Weekly StockCharts snapshot dated 16 November 2012 (open 15.57, high 15.63, low 15.06, last 15.25). Labeled swings keep the printed three-decimal tags; unlabeled in-between prices were read from the candlesticks to about a tenth of a point.

Check the same rules on recent bars

After that multi-year window, the same rules are then examined on the most recent three months so recent signals can be judged separately from the longer sample.

Editorial note: that later-window check is walk-forward analysis. The same mechanical rules are applied to recent bars so entry, exit, and abstention signals are not judged on the full sample alone.

Read the equity path before use

Reviewing equity curves is presented as a check to complete before treating a mechanical procedure as ready to use.

What the check does not settle

Projecting later prices from historical patterns or indicators is described as imperfect, with no guarantee attached to any technology or trading system.

Broad awareness of a method is not treated as automatic failure of that method, because market liquidity is described as large and many competing procedures remain unpublished.

Because markets are described as globally linked and news-sensitive, a static long-term hold is presented as harder to treat as a complete standalone procedure than shorter, predefined entries and exits.

Educational research material, not investment advice. Historical source context does not establish present-day performance.
40 of 51 in the Robustness testing track
201330-33 pp.Next on Robustness testingWalk-forward filter selection for repeated-median velocityThe repeated-median velocity procedure uses three tunable inputs: lookback length N and two signed velocity thresholds that separately gate long and short signals.
All readings on this track · 51 readings
  1. 1986Degrees of freedom in trading system optimization
  2. 1988Walk-forward and neighborhood tests after optimization
  3. 1988Undisclosed rules block system robustness tests
  4. 1988Testing re-optimization calendars against random parameter controls
  5. 1989Binary search limits on multi-peak average grids
  6. 1989Parameter neighborhoods that survive a shift
  7. 1990Use profit mapping to keep a cycle and stop plateau
  8. 1990Why popular indicator optimization fails robustness
  9. 1991Retesting weighted indicator balances across horizons
  10. 1992Constructing forecast models with regression, walk-forward, and robustness
  11. 1992Diagnose regimes before you lock parameters
  12. 1992When stops change system timing
  13. 1993Walk-forward halt rules for forecast models
  14. 1994Walk-forward evaluation of genetic index rules
  15. 1995Input pruning as walk-forward system evaluation
  16. 1995Critiquing neural nets as incomplete trading systems
  17. 1996Rebuild the equity-path ratio before it ranks a designed system
  18. 1996Parameter grids can fit random walks
  19. 1996Walk-forward analysis belongs in the design of a mechanical trading system
  20. 1997When a holdout fails, discard the rule set
  21. 1997Test rewarded rule breaks before replacing the system
  22. 1997Walk-forward rules keep system research from rewriting live trades
  23. 1999Keep a channel-breakout to two lookbacks and test neighbor stability
  24. 1999Constant investment size in stock system evaluation
  25. 2000Forcing optimization maps mechanical system failure boundaries
  26. 2000Robust parameter selection with surface charts
  27. 2001A two-gate classroom test for a two-window momentum trend filter
  28. 2002How a two-sided continuation factor becomes a testable trend rule
  29. 2002Evaluating two-window trend intensity as a reversal rule
  30. 2003Discounting speculative bubbles in system robustness tests
  31. 2003Walk-forward evaluation of locked stochastic oscillator rules
  32. 2003Critiquing mechanical system design after extreme price regimes
  33. 2004Evaluating a two-window trend trigger
  34. 2005Grade backtested signals with holdouts and optimization plateaus
  35. 2006Reserved-sample evaluation of trading system design
  36. 2006Walk-forward critique of hindsight crossover systems
  37. 2008Condition-matched walk-forward evaluation for mechanical systems
  38. 2011Session-split evaluation of regular and overnight systems
  39. 2012Walk-forward evaluation as operator rehearsal
  40. 2013Two-window evaluation of mechanical trading systems
  41. 2013Walk-forward filter selection for repeated-median velocity
  42. 2014Walk-forward evaluation for fading-memory velocity systems
  43. 2015Test oscillator events before tuning rules
  44. 2016Walk-forward evaluation of a five-parameter parabolic stop-and-reversal
  45. 2016Walk-forward optimization without curve fitting
  46. 2017Optimization without overfitting in trend-system evaluation
  47. 2017Parameter stability is a better guide than a larger crossover grid
  48. 2018Point-in-time universes for system evaluation
  49. 2018Walk-forward robustness evaluation for optimized systems
  50. 2018Critiquing breakout systems through robustness tests
  51. 2018A critique of parameter fitting in system design
By fieldSystem design117 readingsBy year201398 readings
All 58 readings tagged Robustness testing
Also on Robustness testing5 readings
1996Walk-forward analysis belongs in the design of a mechanical trading system1997Walk-forward rules keep system research from rewriting live trades2000Forcing optimization maps mechanical system failure boundaries2008Condition-matched walk-forward evaluation for mechanical systems1986Degrees of freedom in trading system optimization