2015issue C0347-54
Event-predictability versus position-constrained rules
System evaluation can be taught as two exams of the same rule set. First rank every defined event on its unoccupied lookahead distribution. Then force those controls through a position-aware engine and a later or opposite-regime sample, where the occupancy-filter, the percent-stop, and walk-forward-analysis can overturn the first ranking.
- Event-predictability ranks a computer-defined occurrence by the center-of-gravity of the percent-change histogram over a fixed number of bars after the event.
- System-optimization scores entry, exit, and abstention together, so a timed-exit, a percent-stop, and an occupancy-filter are part of what gets tested.
- Event-level and trade-level searches typically choose different specifications, because many events are ignored while a position is open and a stop can cut off the lookahead path used in the event score.
- Walk-forward-analysis retests controls chosen in one in-sample-regime on a later or differently shaped sample, including a subset that contains a bear market.
Two exams of the same rule set
A statistical design workflow first builds a dataset of defined events, then tests whether subsequent price change over a fixed bar count has a directional tendency. That measurement is event-predictability: the tendency of a computer-defined occurrence to be followed by a price change over a fixed number of bars.
TradersWeek editorial reading: treat system evaluation as two exams of the same rule set. Rank every defined event first on its unoccupied lookahead distribution. Then force those controls through a position-aware engine and a later or opposite-regime sample, so the occupancy-filter, stops, and walk-forward-analysis can overturn the first ranking.
A close-based stochastic as the defined event
The demonstration event is a close-based stochastic, formed from the latest close relative to the highest and lowest closes in a lookback window.
Several reconstructions enter long on a stochastic threshold cross and exit either after a fixed holding period or when price hits a percentage stop from the entry. Those exits are a timed-exit and a percent-stop.
Shared default controls in several implementations are a lookback of 8, a threshold of 0.3, a 14-bar hold, and a 3.8 percent stop.
How event-predictability is scored
Event scoring looks ahead a set number of bars after each occurrence, bins the percent change, and summarizes the distribution with a center-of-gravity weighted average. A more positive reading is treated as more favorable for long-side use.
A scenario tester can sweep stochastic lookbacks from 8 to 18, thresholds from 0.1 to 0.35 in 0.05 steps, and lookaheads from 5 to 18, then rank combinations by center-of-gravity.
When occupancy and stops change the ranking
Many events are ignored while a position is already open. That occupancy-filter means the engine does not take every occurrence that entered the event-predictability sample.
Stop-loss processing can prevent a taken trade from realizing the lookahead path used in the event score. The percent-stop can close the trade before the fixed bar count that produced the center-of-gravity reading.
System-optimization is a joint search over entry, exit, and abstention controls. The full procedure, not a single indicator setting, is what gets scored.
Occupancy-filtered running balance on Heartland Express

Trade size is one share on 2,800 daily bars. Sixty-eight extra entries were skipped while a long was already open. Trade 130 is still open and marked to 9 January 2015 rather than closed.
A later or opposite-regime retest
One reconstruction warns that a 10-year in-sample optimization window was a strong bull market, so robustness should be rechecked on a different subset that includes a bear market.
Walk-forward-analysis is that later or differently regime-shaped retest of controls chosen on an earlier window. The in-sample-regime used to choose controls may not represent later or opposite market conditions.
The event definition can be swapped
The distribution test can swap the event definition for another rule. A larger higher-frequency sample produces a smoother, more bell-shaped histogram than a smaller sample.
All readings on this track · 50 readings
- 1990Three-window walk-forward system evaluation
- 1990Building the construction layer of a mechanical trading system
- 1991Constructing walk-forward neural trading rules
- 1991Constructing neural trading systems from facts to walk-forward
- 1992Walk-forward evaluation of stop overlays on average crossovers
- 1992Audit mechanical system tests for fills and regimes
- 1993Walk-forward evaluation of monthly yield and real-rate forecasts
- 1993Constructing walk-forward forecasts with linear and moving-average baselines
- 1993Walk-forward hybrid rules for intermarket forecast stacks
- 1994Neural-net construction as a mechanical trading-system problem
- 1995Constructing an intermarket neural net trading system
- 1996Weekly market breadth as one procedure on an unused window
- 1996Walk-forward evaluation of gold-index bond-fund rules
- 1996Evaluating weekday-in-month filters for index day trades
- 1996Require both a trend filter and a cycle oscillator before entry
- 1997Walk-forward windows as a diagnostic of parameter instability
- 1997Walk-forward validation of a market-breadth timing rule
- 1997Sunspot spikes and walk-forward evaluation of an adaptive cycle rule
- 1997A walk-forward check for bond-breadth timing
- 1998Walk-forward audit of regression trend forecasts
- 1998Evaluating a cubic least-squares currency trend with walk-forward segments
- 1998Walk-forward evaluation of recursive yen trend signals
- 1999Personal system design under crowd psychology
- 1999Walk-forward evaluation of a polynomial price forecast
- 2000Walk-forward optimization of regression-slope-angle rules
- 2001Construct a winter seasonal window as one procedure
- 2001Inspectable rules when system write-ups dry up
- 2002Evaluating mechanical systems before position sizing
- 2003Walk-forward construction of rule-based market-position systems
- 2007Evaluating metal seasonal windows across regimes
- 2007Evaluating mechanical timing systems against hold baselines
- 2011Walk-forward reoptimization as a system design gate
- 2011Evaluate generated systems on holdouts, then add stops
- 2012Walk-forward analysis and out-of-sample tests for a mechanical trading system
- 2012Personality-first trading system design
- 2012Scorecard-first mechanical system construction
- 2012Constructing an advancer-decliner moving average for market breadth
- 2012Formula search as mechanical system construction
- 2013Identity-first system construction
- 2013Construct a swing system from bias rules to walk-forward
- 2014Evaluate mechanical stock systems with stops and walk-forward
- 2014Walk-forward velocity filters on noisy intraday trends
- 2015Event-predictability versus position-constrained rules
- 2015Constructing mechanical systems for walk-forward tests
- 2016When a tested system must be retired
- 2016Walk-forward metric filters and chance-level checks for selected inputs
- 2018Evaluate mechanical trading systems without catalog rankings
- 2019Phased stop construction from entry risk to trailing exit
- 2020Stockpiling simple ideas for mechanical system construction
- 2020A pretty first draft is not a walk-forward waiver