Skip to main content
Track Walk-forward analysis
17 / 50
Search the archiveLibrary
Library

1997issue C081-10

Walk-forward validation of a market-breadth timing rule

This archive article restates a 1990s NYSE breadth workflow that used advancing and declining issues, volume, and new highs and new lows. Editorial reading: the case is taught here as a reminder that a market-timing rule is only as trustworthy as the walk-forward procedure that produced it.

  • A relative-strength breadth model can stall in a lockout when the price-strength series reaches its bound before the advance-decline series, leaving the relative measure near zero and blocking later entries.
  • Choosing indicator parameters by searching for the best in-sample result is treated as the same activity as curve-fitting, and an in-sample best fit is not a proof of later signal quality.
  • Showcasing only selected historical episodes, or treating a single optimization over the entire sample as evidence of future behavior, is rejected as validation.
  • After the rules were fixed, later walk-forward windows were inspected on out-of-sample segments, with buy and sell marks overlaid on the S&P 500 and on the breadth series used as inputs.
Entries in this reading3 entries
Current trackWalk-forward analysis95 readingsMarket breadth120 readingsMechanical trading system158 readings

When a relative-strength breadth model locked out

A prior relative-strength breadth model could stall in a lockout when the price-strength series reached its bound before the advance-decline series. The relative measure then sat near zero and later entries were blocked.

A fully mechanical replacement

The replacement model was specified as a fully mechanical procedure. It used advancing and declining issues and volume plus new highs and new lows, rather than a relative-strength difference of two bounded oscillators.

In-sample search is treated as curve-fitting

In this construction, choosing indicator parameters by searching for the best in-sample result is treated as the same activity as curve-fitting. An in-sample best fit is not a proof of later signal quality.

Two common validation shortcuts are rejected. One is showcasing only selected historical episodes. The other is treating a single optimization over the entire sample as evidence of future behavior.

The same historical record, two rule sets

The turbo rule set was compared with the earlier relative-strength version on the same historical record. The newer version avoided the lockout that kept the older version out of the market from early October through year-end 1996.

Walk-forward windows after the rules were fixed

Charts of the later walk-forward windows overlay buy and sell marks on the S&P 500 together with the advancing-declining, volume, and new-high/new-low strength series used as inputs.

Out-of-sample segments were inspected after the rules were fixed. One window covered 1978-82 and used previously unseen data. Another covered 1982-86, in which the model stayed with the dominant advance while losing about 6% in the 1983-84 decline.

In the January 1996-to-publication out-of-sample stretch, the model stayed long except for brief exits around the July 1996 and March 1997 declines.

S&P 500 at each turbo A/D–NH–NL signal, 1978–1997

Each point is an S&P 500 close taken from the article’s trade-by-trade ledger: every turbo-system buy, sell and the 16 May 1997 mark. The 1978–86 walk-forward stretch is choppy and two-sided; after the sixteen parameters were frozen on 1987–95 data the 1996–97 window still rides the bull and only steps aside around the July 1996 and March 1997 dips, finishing at 829.75. These are the printed table values, not a tracing of the raster.
Each point is an S&P 500 close taken from the article’s trade-by-trade ledger: every turbo-system buy, sell and the 16 May 1997 mark. The 1978–86 walk-forward stretch is choppy and two-sided; after the sixteen parameters were frozen on 1987–95 data the 1996–97 window still rides the bull and only steps aside around the July 1996 and March 1997 dips, finishing at 829.75. These are the printed table values, not a tracing of the raster.S&P 500 · Daily · 1978-04-07T00:00:00.000Z to 1997-05-16T00:00:00.000Z

Parameters were fit on 1 Jan 1987–31 Dec 1995. Segment 2 (3 Jan 1978–27 Feb 1987) and segment 3 (1 Jan 1996–16 May 1997) are walk-forward. Sixteen free parameters, so the middle stretch is curve-fit by construction.

Educational research material, not investment advice. Historical source context does not establish present-day performance.
17 of 50 in the Walk-forward analysis track
19971-6 pp.Next on Walk-forward analysisSunspot spikes and walk-forward evaluation of an adaptive cycle ruleDaily sunspot counts from 1983 through 1997 were aligned with continuous S&P 500 and Chicago wheat series so an external cycle measure could be tested against later price paths.
All readings on this track · 50 readings
  1. 1990Three-window walk-forward system evaluation
  2. 1990Building the construction layer of a mechanical trading system
  3. 1991Constructing walk-forward neural trading rules
  4. 1991Constructing neural trading systems from facts to walk-forward
  5. 1992Walk-forward evaluation of stop overlays on average crossovers
  6. 1992Audit mechanical system tests for fills and regimes
  7. 1993Walk-forward evaluation of monthly yield and real-rate forecasts
  8. 1993Constructing walk-forward forecasts with linear and moving-average baselines
  9. 1993Walk-forward hybrid rules for intermarket forecast stacks
  10. 1994Neural-net construction as a mechanical trading-system problem
  11. 1995Constructing an intermarket neural net trading system
  12. 1996Weekly market breadth as one procedure on an unused window
  13. 1996Walk-forward evaluation of gold-index bond-fund rules
  14. 1996Evaluating weekday-in-month filters for index day trades
  15. 1996Require both a trend filter and a cycle oscillator before entry
  16. 1997Walk-forward windows as a diagnostic of parameter instability
  17. 1997Walk-forward validation of a market-breadth timing rule
  18. 1997Sunspot spikes and walk-forward evaluation of an adaptive cycle rule
  19. 1997A walk-forward check for bond-breadth timing
  20. 1998Walk-forward audit of regression trend forecasts
  21. 1998Evaluating a cubic least-squares currency trend with walk-forward segments
  22. 1998Walk-forward evaluation of recursive yen trend signals
  23. 1999Personal system design under crowd psychology
  24. 1999Walk-forward evaluation of a polynomial price forecast
  25. 2000Walk-forward optimization of regression-slope-angle rules
  26. 2001Construct a winter seasonal window as one procedure
  27. 2001Inspectable rules when system write-ups dry up
  28. 2002Evaluating mechanical systems before position sizing
  29. 2003Walk-forward construction of rule-based market-position systems
  30. 2007Evaluating metal seasonal windows across regimes
  31. 2007Evaluating mechanical timing systems against hold baselines
  32. 2011Walk-forward reoptimization as a system design gate
  33. 2011Evaluate generated systems on holdouts, then add stops
  34. 2012Walk-forward analysis and out-of-sample tests for a mechanical trading system
  35. 2012Personality-first trading system design
  36. 2012Scorecard-first mechanical system construction
  37. 2012Constructing an advancer-decliner moving average for market breadth
  38. 2012Formula search as mechanical system construction
  39. 2013Identity-first system construction
  40. 2013Construct a swing system from bias rules to walk-forward
  41. 2014Evaluate mechanical stock systems with stops and walk-forward
  42. 2014Walk-forward velocity filters on noisy intraday trends
  43. 2015Event-predictability versus position-constrained rules
  44. 2015Constructing mechanical systems for walk-forward tests
  45. 2016When a tested system must be retired
  46. 2016Walk-forward metric filters and chance-level checks for selected inputs
  47. 2018Evaluate mechanical trading systems without catalog rankings
  48. 2019Phased stop construction from entry risk to trailing exit
  49. 2020Stockpiling simple ideas for mechanical system construction
  50. 2020A pretty first draft is not a walk-forward waiver
By fieldMarket breadth101 readingsBy year199756 readings
All 95 readings tagged Walk-forward analysis
Also on Walk-forward analysis5 readings
1996Weekly market breadth as one procedure on an unused window1997A walk-forward check for bond-breadth timing2012Constructing an advancer-decliner moving average for market breadth1990Building the construction layer of a mechanical trading system1991Constructing neural trading systems from facts to walk-forward