Split-sample tests for stochastic parameter grids

Primary parameters and the trading plan

A mechanical plan can be separated into three pieces: a stipulated increment of price or time, a count of those increments used as lookback, and a trading plan that acts on the resulting series. The increment is the gang-size inside which market movement is compared. Increment and lookback were labeled primary-parameter settings because they can be chosen before any particular trading plan is specified.

A middle path for the series

A stochastic construction that uses high, low, and close, while adding only one plan-level smoothing setting, was chosen as a middle path. It sits between a close-only moving-average plan and models that pull in every available field. That leftover smoothing choice is a system-parameter. It belongs to the trading plan rather than to the sampling grid.

The locked crossover plan

The tested plan was a stochastic-crossover-plan. It stayed continuously long or short and reversed when the faster stochastic line crossed the slower line. No commission or slippage was subtracted. Increment was stepped from a daily unit to a monthly unit by two-day steps. Lookback ran from 3 through 36 bars. The three-period double smoothing that produces the slower line was never varied.

Half-sample-split and the ranking statistic

Twenty-nine commodity series, most spanning about 10 to 20 years, were each divided by a half-sample-split into two contiguous blocks. Each half was searched for preferred increment and bar count. The ranking statistic was percent-return-on-equity. A transfer-test then checked whether a setting preferred on one block remained preferred, or even informative, on the unused block.

Transfer-test on the unused half

Scatter of first-half preferred increment against second-half preferred increment showed no discernible alignment with the identity line. The same increment's equity-return reading on one half did not indicate the reading obtained with that increment on the other half. The bar count that ranked first inside a given increment on one half did not identify the bar count that ranked first on the other half.

The split-sample pattern was treated as reason to doubt that a stochastic-parameter search will identify settings that remain preferred later. The design was described as unable to prove that such searching never works.