Degrees of freedom in trading system optimization

How the in-sample search treats an optimum

System-optimization searches over rule inputs, market state, and execution constraints so entry, exit, and abstention become one testable procedure whose output is a holding-period signal. Searching three moving-average controls at ten candidate values each requires evaluating 1,000 combinations, after which the most profitable set, and often the one with the smallest drawdown, is commonly treated as an optimum.

Degrees-of-freedom-loss from extra controls

Each added control parameter consumes statistical freedom and reduces the reliability of the simulated outcome. That reliability penalty is degrees-of-freedom-loss. A ninth-degree polynomial forced through ten closing prices can show a correlation of 1.0 and still fail to forecast the next close because the fit has used its available freedom.

Adding more averages, stops, range filters, deviation bands, and oscillator thresholds increases control and generally lowers reliability unless joint correlation among inputs is tested and redundant variables are dropped. Optional entry timing among the open, the close, or an in-between print can consume freedom without a commensurate design benefit.

Joint-correlation and redundant inputs

Joint-correlation is association among candidate inputs that can overstate simulated outcomes unless redundant variables are removed. If volume and daily range move together, only one should enter system-optimization, because two correlated inputs describe the same condition and can overstate the result.

Models that forecast activity from many mutually independent economic series are distinguished from predicting future price from past price, so overlapping price-derived controls should be reduced to a small independent set.

Minimum-trade-sample and cycle-complete windows

A one-year, heavily parameterized simulation that produced 19 winning trades and one losing trade later produced 19 losing trades and one winning trade across about 20 subsequent trades after too much control was applied to too short a history.

Preferred designs keep about two to five variables and generate 30 or more simulated trades, using that 30-trade floor as the sampling-theory threshold for approaching normality. That count is the minimum-trade-sample.

The simulation window should be an integer multiple of a full low-frequency cycle. A cycle-complete-window is that span, chosen to avoid a buy or sell bias. A five-year test against a four-year hog cycle, or a four-month test against a three-month cycle, can inject directional bias unless the span is extended to at least six months in the three-month-cycle case.

Robustness-testing then walk-forward-analysis

Robustness-testing checks whether that signal still holds after parameter load, input independence, trade-sample size, and test-window design are examined for overstated results.

Walk-forward-analysis reapplies the same optimization procedure on later market state so selected rules are not judged only on the window that produced them.