Skip to main content
Track Robustness testing
11 / 51
Search the archiveLibrary
Library

1992issue C071-10

Diagnose regimes before you lock parameters

Choosing the numeric controls inside a signal rule is treated as system optimization when entry, exit, and stand-aside rules are assigned as one procedure. Peak historical settings and unchecked values are treated as inadequate unless the contract month is first named as linear, periodic, or random motion.

  • System optimization assigns every indicator control so entry, exit, and stand-aside rules can be tested as one procedure rather than as isolated knobs.
  • Selecting the historically most profitable setting and assigning values without a validity check are both treated as inadequate ways to set parameters.
  • Markets mix linear motion, periodic motion, and random motion, and no single technical approach is said to hold when a different one of those behaviors dominates.
  • Regular re-tuning and user-selectable modes stay incomplete because the next regime is not known in advance.
Entries in this reading3 entries
Current trackRobustness testing58 readingsSystem optimization85 readingsWalk-forward analysis95 readings

Parameter choice is a full procedure

Signal-generating indicators contain one or more parameters. A parameter is a numeric control inside a signal rule whose value changes when and how trades fire. Choosing those numeric settings is the act treated as system optimization.

System optimization assigns every indicator control so entry, exit, and stand-aside rules can be tested as one procedure rather than as isolated knobs.

Peak profit and unchecked values are not enough

Selecting the historically most profitable setting and assigning values without any validity check are both treated as inadequate ways to set parameters.

Unverified assignment gives parameters values without checking that those values correspond to a repeating market structure. A backtest that maximizes past profit can, at the extreme, encode a one-day hindsight rule that cannot function as a forecast.

Hindsight fitting selects settings because they describe prices that have already printed, including rules that could only be written after the fact.

Slope, cycle, and noise do not share a rule family

Markets are described as mixing linear motion, periodic motion, and random motion. No single technical approach is said to hold when a different one of those behaviors dominates.

Linear motion is persistent price change along a slope, the condition associated with trend-following rules. Moving-average penetration and similar trend-following rules are presented as matched to linear motion and poorly matched to cyclical stretches.

Periodic motion is repeating cyclical or range-bound action, the condition associated with oscillator-style rules. Oscillator-style rules are presented as the reverse of that match.

Random motion is price action without a usable slope or cycle, where standard technical indicators are treated as unreliable. Random-looking price action is presented as a condition in which both trend-following and contrarian indicators can fail together.

May 1989 wheat slow stochastic, 10-period %K

The 10-period slow %K swings through the 20 and 80 bands again and again from October through February, which is the cycle regime in which the article’s oscillator entry and exit locks make sense. Dates and percent readings were taken off the printed daily pane, so they are approximate.
The 10-period slow %K swings through the 20 and 80 bands again and again from October through February, which is the cycle regime in which the article’s oscillator entry and exit locks make sense. Dates and percent readings were taken off the printed daily pane, so they are approximate.May 1989 wheat futures · daily · 1988-09-20T00:00:00.000Z to 1989-02-22T00:00:00.000Z

The source fixes slow stochastic at 10-period %K and 3-period %D and applies buy below 20 / sell above 80. Calendar dates are interpolated from the monthly axis of a daily chart; the two printed lines overlap too tightly to separate %D honestly.

Seasonal windows and delivery months are separate questions

A recurring seasonal window can show more trending than ranging or random episodes without a reliable directional bias. That pattern is seasonal volatility, and it is distinguished from a seasonal directional trade. Seasonal periodicity is a calendar-linked recurrence of cyclical rather than trending behavior.

Different delivery months of the same commodity can require separate systems because their fundamentals, and therefore their periodicity, need not match. The contract-month market view treats each delivery month as its own market because its drivers need not match other months of the same commodity.

Reactive modes do not complete the exam

Regular re-tuning and user-selectable modes are treated as incomplete optimization because the change is reactive and the trader cannot know in advance which mode will match the next regime.

Mode packaging exposes several internal rule sets as user-chosen modes so that at least one path can look suitable after the fact.

Educational research material, not investment advice. Historical source context does not establish present-day performance.
11 of 51 in the Robustness testing track
19921-11 pp.Next on Robustness testingWhen stops change system timingAfter a long trade is closed at a fixed dollar target, a later long signal may be taken that would have been ignored if the first position had still been open.
All readings on this track · 51 readings
  1. 1986Degrees of freedom in trading system optimization
  2. 1988Walk-forward and neighborhood tests after optimization
  3. 1988Undisclosed rules block system robustness tests
  4. 1988Testing re-optimization calendars against random parameter controls
  5. 1989Binary search limits on multi-peak average grids
  6. 1989Parameter neighborhoods that survive a shift
  7. 1990Use profit mapping to keep a cycle and stop plateau
  8. 1990Why popular indicator optimization fails robustness
  9. 1991Retesting weighted indicator balances across horizons
  10. 1992Constructing forecast models with regression, walk-forward, and robustness
  11. 1992Diagnose regimes before you lock parameters
  12. 1992When stops change system timing
  13. 1993Walk-forward halt rules for forecast models
  14. 1994Walk-forward evaluation of genetic index rules
  15. 1995Input pruning as walk-forward system evaluation
  16. 1995Critiquing neural nets as incomplete trading systems
  17. 1996Rebuild the equity-path ratio before it ranks a designed system
  18. 1996Parameter grids can fit random walks
  19. 1996Walk-forward analysis belongs in the design of a mechanical trading system
  20. 1997When a holdout fails, discard the rule set
  21. 1997Test rewarded rule breaks before replacing the system
  22. 1997Walk-forward rules keep system research from rewriting live trades
  23. 1999Keep a channel-breakout to two lookbacks and test neighbor stability
  24. 1999Constant investment size in stock system evaluation
  25. 2000Forcing optimization maps mechanical system failure boundaries
  26. 2000Robust parameter selection with surface charts
  27. 2001A two-gate classroom test for a two-window momentum trend filter
  28. 2002How a two-sided continuation factor becomes a testable trend rule
  29. 2002Evaluating two-window trend intensity as a reversal rule
  30. 2003Discounting speculative bubbles in system robustness tests
  31. 2003Walk-forward evaluation of locked stochastic oscillator rules
  32. 2003Critiquing mechanical system design after extreme price regimes
  33. 2004Evaluating a two-window trend trigger
  34. 2005Grade backtested signals with holdouts and optimization plateaus
  35. 2006Reserved-sample evaluation of trading system design
  36. 2006Walk-forward critique of hindsight crossover systems
  37. 2008Condition-matched walk-forward evaluation for mechanical systems
  38. 2011Session-split evaluation of regular and overnight systems
  39. 2012Walk-forward evaluation as operator rehearsal
  40. 2013Two-window evaluation of mechanical trading systems
  41. 2013Walk-forward filter selection for repeated-median velocity
  42. 2014Walk-forward evaluation for fading-memory velocity systems
  43. 2015Test oscillator events before tuning rules
  44. 2016Walk-forward evaluation of a five-parameter parabolic stop-and-reversal
  45. 2016Walk-forward optimization without curve fitting
  46. 2017Optimization without overfitting in trend-system evaluation
  47. 2017Parameter stability is a better guide than a larger crossover grid
  48. 2018Point-in-time universes for system evaluation
  49. 2018Walk-forward robustness evaluation for optimized systems
  50. 2018Critiquing breakout systems through robustness tests
  51. 2018A critique of parameter fitting in system design
By fieldSystem design117 readingsBy year1992115 readings
All 58 readings tagged Robustness testing
Also on Robustness testing5 readings
1986Degrees of freedom in trading system optimization1988Testing re-optimization calendars against random parameter controls1988Walk-forward and neighborhood tests after optimization1993Walk-forward halt rules for forecast models1994Walk-forward evaluation of genetic index rules