Testing re-optimization calendars against random parameter controls

The calendar sits inside the procedure

System-optimization chooses a system's lookback or threshold from ranked historical results instead of committing to a setting before the test begins. The re-optimization calendar is the joint choice of how much past data ranks the next parameter and how long that choice is held before the search is allowed to run again.

That calendar decides when a new lookback may be selected and which already observed results are allowed to rank the candidates. It is part of the trading procedure, not a setting that can be left outside the test.

What the historical evaluation compared

A historical evaluation compared several history-based re-optimization calendars with a random-parameter-control across a one-parameter channel breakout and a directional-movement index crossover. The random-parameter-control assigns each evaluation period's setting by chance so the search procedure itself can be tested.

Practitioners at the time did not share a standard for how much history to use, how often to refresh parameters, or whether to search at all. Lookbacks ranged from a few years to all available data, and refresh intervals ranged from twice a year to once every several years.

Each system was searched over a parameter-grid spanning short to multi-month lookbacks in even steps. Both systems were simulated on a multi-market futures portfolio, with equal margin allocations, a fixed commission per trade, and positions taken only in the nearby contract.

Walk-forward choice and hold periods

Walk-forward analysis selects a parameter on an earlier window and applies that choice only on a later interval that was not used to rank the candidates. Every history-based calendar chose the next parameter only from already observed results, so the evaluation interval never reused the observations that ranked the candidates.

Annual-refresh calendars used several prior years of profit ranking, or all history from market inception or a fixed early start. Fixed calendars held a multi-year choice for a matching multi-year block.

Paired tests including the random control

Robustness-testing checks whether alternative search calendars, including a chance-assignment control, produce statistically distinguishable later results. Statistical comparison used tests of mean portfolio results against zero and paired-difference tests among calendars, including the random-parameter-control.

On the breakout system, paired-difference tests found no statistically significant gap among mean monthly results of the calendars, including the gap versus random assignment.

On the directional-movement system, only a few pairwise contrasts reached significance, no calendar formed a consistent ranking, and none differed statistically from random assignment.

Within the tested systems, parameter-grids, and calendars, history-based re-optimization did not produce a statistically greater result than chance assignment and did not statistically impair mean results.

What the in-sample champion cannot replace

The in-sample-champion is the single historically best setting on a window. Its own window result is not a valid estimate of later results, so that champion is not a valid stand-in for what follows. The center of the full tested parameter-grid is the more defensible historical benchmark.