Skip to main content
Track Robustness testing
43 / 51
Library

2015issue C0328-31

Test oscillator events before tuning rules

A coded stochastic crossing can be scored by the later close-to-close change before any full entry-exit procedure is written. After that event is treated as informative, lookback, threshold, holding period, and a percent loss limit can be adjusted as one procedure, then the same scoring bed can be reused on other symbols and bar clocks.

  • Begin system design by asking whether an identifiable event has a statistical association with later prices, before a full entry-exit procedure is written.
  • A stochastic oscillator places the close between the highest and lowest closes over a lookback and can mark a long-side event when the reading crosses below a low threshold.
  • Lookback-shifted scoring records that event on a past bar, bins the clipped percent close-to-close change, and summarizes typical later change with a center-of-gravity reading.
  • Only after the event is treated as informative should lookback, threshold, holding period, and a percent loss limit be adjusted as one procedure, then repeated on other symbols and bar intervals.
Entries in this reading3 entries

Ask about later prices first

Oscillator-style tools can be described as first-order high-pass filters that remove longer price swings, while moving averages can be described as smoothers that remove high-frequency variation.

A system-design sequence can begin by asking whether a coded, identifiable event has a statistical association with later prices, before a full entry-exit procedure is written. An identifiable event is any condition that can be written as true or false on a historical bar and then scored by the subsequent percent change in price.

Mark a stochastic crossing as an event

A stochastic oscillator is a unit-scaled placement of the close between the highest and lowest closes over a lookback, used here to mark a coded event when the reading crosses below a low threshold. Built that way from the close relative to the highest and lowest closes over the lookback, it can mark a long-side event at that downward cross.

Score the later close-to-close change

Lookback-shifted scoring records the event on a past bar and measures the percent close-to-close change from that bar to the present so the subsequent path is already observed. One evaluation design records the event on a bar 10 bars in the past and measures the percent close-to-close change from that bar to the current bar.

Subsequent-return bins are a fixed grid of clipped percent changes used to accumulate an empirical distribution of outcomes after events. In that procedure, the subsequent percent change is clipped to a minus-10 to plus-10 percent band, rescaled onto a 0 to 100 axis, and counted into 100 bins to form an empirical outcome distribution.

A center-of-gravity calculation on those bin counts is used as a compact location summary of typical subsequent percent change. Center of gravity here means a first-moment location of the binned outcome counts, recentered so a balanced outline sits near a no-change reading.

Trim the procedure only after the event is scored

After an event is treated as informative, the same oscillator can be rewritten as one procedure whose lookback, threshold, holding period, and percent loss limit are adjustable. System optimization is that later step: adjusting lookback, threshold, holding period, and a percent loss limit only after an event has already been scored by the distribution of subsequent prices.

The same event-testing bed can be reused across symbols and at different bar intervals, including intraday or equal-tick sampling. Robustness testing is repeating the same event-scoring procedure on other symbols and other bar intervals to see whether the event remains informative.

Educational research material, not investment advice. Historical source context does not establish present-day performance.
43 of 51 in the Robustness testing track
201626-29 pp.Next on Robustness testingWalk-forward evaluation of a five-parameter parabolic stop-and-reversalStarting acceleration, increment, maximum acceleration, a noise-crossover amount, and an initial-stop offset can be searched together as one five-parameter parabolic stop-and-reversal procedure.
All readings on this track · 51 readings
  1. 1986Degrees of freedom in trading system optimization
  2. 1988Walk-forward and neighborhood tests after optimization
  3. 1988Undisclosed rules block system robustness tests
  4. 1988Testing re-optimization calendars against random parameter controls
  5. 1989Binary search limits on multi-peak average grids
  6. 1989Parameter neighborhoods that survive a shift
  7. 1990Use profit mapping to keep a cycle and stop plateau
  8. 1990Why popular indicator optimization fails robustness
  9. 1991Retesting weighted indicator balances across horizons
  10. 1992Constructing forecast models with regression, walk-forward, and robustness
  11. 1992Diagnose regimes before you lock parameters
  12. 1992When stops change system timing
  13. 1993Walk-forward halt rules for forecast models
  14. 1994Walk-forward evaluation of genetic index rules
  15. 1995Input pruning as walk-forward system evaluation
  16. 1995Critiquing neural nets as incomplete trading systems
  17. 1996Rebuild the equity-path ratio before it ranks a designed system
  18. 1996Parameter grids can fit random walks
  19. 1996Walk-forward analysis belongs in the design of a mechanical trading system
  20. 1997When a holdout fails, discard the rule set
  21. 1997Test rewarded rule breaks before replacing the system
  22. 1997Walk-forward rules keep system research from rewriting live trades
  23. 1999Keep a channel-breakout to two lookbacks and test neighbor stability
  24. 1999Constant investment size in stock system evaluation
  25. 2000Forcing optimization maps mechanical system failure boundaries
  26. 2000Robust parameter selection with surface charts
  27. 2001A two-gate classroom test for a two-window momentum trend filter
  28. 2002How a two-sided continuation factor becomes a testable trend rule
  29. 2002Evaluating two-window trend intensity as a reversal rule
  30. 2003Discounting speculative bubbles in system robustness tests
  31. 2003Walk-forward evaluation of locked stochastic oscillator rules
  32. 2003Critiquing mechanical system design after extreme price regimes
  33. 2004Evaluating a two-window trend trigger
  34. 2005Grade backtested signals with holdouts and optimization plateaus
  35. 2006Reserved-sample evaluation of trading system design
  36. 2006Walk-forward critique of hindsight crossover systems
  37. 2008Condition-matched walk-forward evaluation for mechanical systems
  38. 2011Session-split evaluation of regular and overnight systems
  39. 2012Walk-forward evaluation as operator rehearsal
  40. 2013Two-window evaluation of mechanical trading systems
  41. 2013Walk-forward filter selection for repeated-median velocity
  42. 2014Walk-forward evaluation for fading-memory velocity systems
  43. 2015Test oscillator events before tuning rules
  44. 2016Walk-forward evaluation of a five-parameter parabolic stop-and-reversal
  45. 2016Walk-forward optimization without curve fitting
  46. 2017Optimization without overfitting in trend-system evaluation
  47. 2017Parameter stability is a better guide than a larger crossover grid
  48. 2018Point-in-time universes for system evaluation
  49. 2018Walk-forward robustness evaluation for optimized systems
  50. 2018Critiquing breakout systems through robustness tests
  51. 2018A critique of parameter fitting in system design
All 58 readings tagged Robustness testing
Also on Robustness testing5 readings