Skip to main content
Track Walk-forward analysis
43 / 50
Library

2015issue C0347-54

Event-predictability versus position-constrained rules

System evaluation can be taught as two exams of the same rule set. First rank every defined event on its unoccupied lookahead distribution. Then force those controls through a position-aware engine and a later or opposite-regime sample, where the occupancy-filter, the percent-stop, and walk-forward-analysis can overturn the first ranking.

  • Event-predictability ranks a computer-defined occurrence by the center-of-gravity of the percent-change histogram over a fixed number of bars after the event.
  • System-optimization scores entry, exit, and abstention together, so a timed-exit, a percent-stop, and an occupancy-filter are part of what gets tested.
  • Event-level and trade-level searches typically choose different specifications, because many events are ignored while a position is open and a stop can cut off the lookahead path used in the event score.
  • Walk-forward-analysis retests controls chosen in one in-sample-regime on a later or differently shaped sample, including a subset that contains a bear market.
Entries in this reading2 entries

Two exams of the same rule set

A statistical design workflow first builds a dataset of defined events, then tests whether subsequent price change over a fixed bar count has a directional tendency. That measurement is event-predictability: the tendency of a computer-defined occurrence to be followed by a price change over a fixed number of bars.

TradersWeek editorial reading: treat system evaluation as two exams of the same rule set. Rank every defined event first on its unoccupied lookahead distribution. Then force those controls through a position-aware engine and a later or opposite-regime sample, so the occupancy-filter, stops, and walk-forward-analysis can overturn the first ranking.

A close-based stochastic as the defined event

The demonstration event is a close-based stochastic, formed from the latest close relative to the highest and lowest closes in a lookback window.

Several reconstructions enter long on a stochastic threshold cross and exit either after a fixed holding period or when price hits a percentage stop from the entry. Those exits are a timed-exit and a percent-stop.

Shared default controls in several implementations are a lookback of 8, a threshold of 0.3, a 14-bar hold, and a 3.8 percent stop.

How event-predictability is scored

Event scoring looks ahead a set number of bars after each occurrence, bins the percent change, and summarizes the distribution with a center-of-gravity weighted average. A more positive reading is treated as more favorable for long-side use.

A scenario tester can sweep stochastic lookbacks from 8 to 18, thresholds from 0.1 to 0.35 in 0.05 steps, and lookaheads from 5 to 18, then rank combinations by center-of-gravity.

When occupancy and stops change the ranking

Many events are ignored while a position is already open. That occupancy-filter means the engine does not take every occurrence that entered the event-predictability sample.

Stop-loss processing can prevent a taken trade from realizing the lookahead path used in the event score. The percent-stop can close the trade before the fixed bar count that produced the center-of-gravity reading.

System-optimization is a joint search over entry, exit, and abstention controls. The full procedure, not a single indicator setting, is what gets scored.

Occupancy-filtered running balance on Heartland Express

After the stochastic-cross rule is limited to one long at a time, the last 19 trades of the Heartland Express test still lift a one-share book from 21.68 to 32.87 dollars. The November 2013 winner of +3.19 is the obvious step in this window. Every point is the running-balance column of the transaction-summary table covering 25 November 2003 through 9 January 2015, a 130-trade run that finished 71 winners against 59 losers.
After the stochastic-cross rule is limited to one long at a time, the last 19 trades of the Heartland Express test still lift a one-share book from 21.68 to 32.87 dollars. The November 2013 winner of +3.19 is the obvious step in this window. Every point is the running-balance column of the transaction-summary table covering 25 November 2003 through 9 January 2015, a 130-trade run that finished 71 winners against 59 losers.Heartland Express (HTLD) · Daily · 2003-11-25T00:00:00.000Z to 2015-01-09T00:00:00.000Z

Trade size is one share on 2,800 daily bars. Sixty-eight extra entries were skipped while a long was already open. Trade 130 is still open and marked to 9 January 2015 rather than closed.

A later or opposite-regime retest

One reconstruction warns that a 10-year in-sample optimization window was a strong bull market, so robustness should be rechecked on a different subset that includes a bear market.

Walk-forward-analysis is that later or differently regime-shaped retest of controls chosen on an earlier window. The in-sample-regime used to choose controls may not represent later or opposite market conditions.

The event definition can be swapped

The distribution test can swap the event definition for another rule. A larger higher-frequency sample produces a smoother, more bell-shaped histogram than a smaller sample.

Educational research material, not investment advice. Historical source context does not establish present-day performance.
43 of 50 in the Walk-forward analysis track
201528-33 pp.Next on Walk-forward analysisConstructing mechanical systems for walk-forward testsPrice-series choice comes before rule search, because a series that is too violent leaves no safe mechanical procedure and a series that is too quiet lacks usable opportunity.
All readings on this track · 50 readings
  1. 1990Three-window walk-forward system evaluation
  2. 1990Building the construction layer of a mechanical trading system
  3. 1991Constructing walk-forward neural trading rules
  4. 1991Constructing neural trading systems from facts to walk-forward
  5. 1992Walk-forward evaluation of stop overlays on average crossovers
  6. 1992Audit mechanical system tests for fills and regimes
  7. 1993Walk-forward evaluation of monthly yield and real-rate forecasts
  8. 1993Constructing walk-forward forecasts with linear and moving-average baselines
  9. 1993Walk-forward hybrid rules for intermarket forecast stacks
  10. 1994Neural-net construction as a mechanical trading-system problem
  11. 1995Constructing an intermarket neural net trading system
  12. 1996Weekly market breadth as one procedure on an unused window
  13. 1996Walk-forward evaluation of gold-index bond-fund rules
  14. 1996Evaluating weekday-in-month filters for index day trades
  15. 1996Require both a trend filter and a cycle oscillator before entry
  16. 1997Walk-forward windows as a diagnostic of parameter instability
  17. 1997Walk-forward validation of a market-breadth timing rule
  18. 1997Sunspot spikes and walk-forward evaluation of an adaptive cycle rule
  19. 1997A walk-forward check for bond-breadth timing
  20. 1998Walk-forward audit of regression trend forecasts
  21. 1998Evaluating a cubic least-squares currency trend with walk-forward segments
  22. 1998Walk-forward evaluation of recursive yen trend signals
  23. 1999Personal system design under crowd psychology
  24. 1999Walk-forward evaluation of a polynomial price forecast
  25. 2000Walk-forward optimization of regression-slope-angle rules
  26. 2001Construct a winter seasonal window as one procedure
  27. 2001Inspectable rules when system write-ups dry up
  28. 2002Evaluating mechanical systems before position sizing
  29. 2003Walk-forward construction of rule-based market-position systems
  30. 2007Evaluating metal seasonal windows across regimes
  31. 2007Evaluating mechanical timing systems against hold baselines
  32. 2011Walk-forward reoptimization as a system design gate
  33. 2011Evaluate generated systems on holdouts, then add stops
  34. 2012Walk-forward analysis and out-of-sample tests for a mechanical trading system
  35. 2012Personality-first trading system design
  36. 2012Scorecard-first mechanical system construction
  37. 2012Constructing an advancer-decliner moving average for market breadth
  38. 2012Formula search as mechanical system construction
  39. 2013Identity-first system construction
  40. 2013Construct a swing system from bias rules to walk-forward
  41. 2014Evaluate mechanical stock systems with stops and walk-forward
  42. 2014Walk-forward velocity filters on noisy intraday trends
  43. 2015Event-predictability versus position-constrained rules
  44. 2015Constructing mechanical systems for walk-forward tests
  45. 2016When a tested system must be retired
  46. 2016Walk-forward metric filters and chance-level checks for selected inputs
  47. 2018Evaluate mechanical trading systems without catalog rankings
  48. 2019Phased stop construction from entry risk to trailing exit
  49. 2020Stockpiling simple ideas for mechanical system construction
  50. 2020A pretty first draft is not a walk-forward waiver
All 95 readings tagged Walk-forward analysis
Also on Walk-forward analysis5 readings