2015issue C0328-31
Test oscillator events before tuning rules
A coded stochastic crossing can be scored by the later close-to-close change before any full entry-exit procedure is written. After that event is treated as informative, lookback, threshold, holding period, and a percent loss limit can be adjusted as one procedure, then the same scoring bed can be reused on other symbols and bar clocks.
- Begin system design by asking whether an identifiable event has a statistical association with later prices, before a full entry-exit procedure is written.
- A stochastic oscillator places the close between the highest and lowest closes over a lookback and can mark a long-side event when the reading crosses below a low threshold.
- Lookback-shifted scoring records that event on a past bar, bins the clipped percent close-to-close change, and summarizes typical later change with a center-of-gravity reading.
- Only after the event is treated as informative should lookback, threshold, holding period, and a percent loss limit be adjusted as one procedure, then repeated on other symbols and bar intervals.
Ask about later prices first
Oscillator-style tools can be described as first-order high-pass filters that remove longer price swings, while moving averages can be described as smoothers that remove high-frequency variation.
A system-design sequence can begin by asking whether a coded, identifiable event has a statistical association with later prices, before a full entry-exit procedure is written. An identifiable event is any condition that can be written as true or false on a historical bar and then scored by the subsequent percent change in price.
Mark a stochastic crossing as an event
A stochastic oscillator is a unit-scaled placement of the close between the highest and lowest closes over a lookback, used here to mark a coded event when the reading crosses below a low threshold. Built that way from the close relative to the highest and lowest closes over the lookback, it can mark a long-side event at that downward cross.
Score the later close-to-close change
Lookback-shifted scoring records the event on a past bar and measures the percent close-to-close change from that bar to the present so the subsequent path is already observed. One evaluation design records the event on a bar 10 bars in the past and measures the percent close-to-close change from that bar to the current bar.
Subsequent-return bins are a fixed grid of clipped percent changes used to accumulate an empirical distribution of outcomes after events. In that procedure, the subsequent percent change is clipped to a minus-10 to plus-10 percent band, rescaled onto a 0 to 100 axis, and counted into 100 bins to form an empirical outcome distribution.
A center-of-gravity calculation on those bin counts is used as a compact location summary of typical subsequent percent change. Center of gravity here means a first-moment location of the binned outcome counts, recentered so a balanced outline sits near a no-change reading.
Trim the procedure only after the event is scored
After an event is treated as informative, the same oscillator can be rewritten as one procedure whose lookback, threshold, holding period, and percent loss limit are adjustable. System optimization is that later step: adjusting lookback, threshold, holding period, and a percent loss limit only after an event has already been scored by the distribution of subsequent prices.
The same event-testing bed can be reused across symbols and at different bar intervals, including intraday or equal-tick sampling. Robustness testing is repeating the same event-scoring procedure on other symbols and other bar intervals to see whether the event remains informative.
All readings on this track · 51 readings
- 1986Degrees of freedom in trading system optimization
- 1988Walk-forward and neighborhood tests after optimization
- 1988Undisclosed rules block system robustness tests
- 1988Testing re-optimization calendars against random parameter controls
- 1989Binary search limits on multi-peak average grids
- 1989Parameter neighborhoods that survive a shift
- 1990Use profit mapping to keep a cycle and stop plateau
- 1990Why popular indicator optimization fails robustness
- 1991Retesting weighted indicator balances across horizons
- 1992Constructing forecast models with regression, walk-forward, and robustness
- 1992Diagnose regimes before you lock parameters
- 1992When stops change system timing
- 1993Walk-forward halt rules for forecast models
- 1994Walk-forward evaluation of genetic index rules
- 1995Input pruning as walk-forward system evaluation
- 1995Critiquing neural nets as incomplete trading systems
- 1996Rebuild the equity-path ratio before it ranks a designed system
- 1996Parameter grids can fit random walks
- 1996Walk-forward analysis belongs in the design of a mechanical trading system
- 1997When a holdout fails, discard the rule set
- 1997Test rewarded rule breaks before replacing the system
- 1997Walk-forward rules keep system research from rewriting live trades
- 1999Keep a channel-breakout to two lookbacks and test neighbor stability
- 1999Constant investment size in stock system evaluation
- 2000Forcing optimization maps mechanical system failure boundaries
- 2000Robust parameter selection with surface charts
- 2001A two-gate classroom test for a two-window momentum trend filter
- 2002How a two-sided continuation factor becomes a testable trend rule
- 2002Evaluating two-window trend intensity as a reversal rule
- 2003Discounting speculative bubbles in system robustness tests
- 2003Walk-forward evaluation of locked stochastic oscillator rules
- 2003Critiquing mechanical system design after extreme price regimes
- 2004Evaluating a two-window trend trigger
- 2005Grade backtested signals with holdouts and optimization plateaus
- 2006Reserved-sample evaluation of trading system design
- 2006Walk-forward critique of hindsight crossover systems
- 2008Condition-matched walk-forward evaluation for mechanical systems
- 2011Session-split evaluation of regular and overnight systems
- 2012Walk-forward evaluation as operator rehearsal
- 2013Two-window evaluation of mechanical trading systems
- 2013Walk-forward filter selection for repeated-median velocity
- 2014Walk-forward evaluation for fading-memory velocity systems
- 2015Test oscillator events before tuning rules
- 2016Walk-forward evaluation of a five-parameter parabolic stop-and-reversal
- 2016Walk-forward optimization without curve fitting
- 2017Optimization without overfitting in trend-system evaluation
- 2017Parameter stability is a better guide than a larger crossover grid
- 2018Point-in-time universes for system evaluation
- 2018Walk-forward robustness evaluation for optimized systems
- 2018Critiquing breakout systems through robustness tests
- 2018A critique of parameter fitting in system design