Walk-forward halt rules for forecast models

Training versus walk-forward recall

Training updates internal weights, while walk-forward recall feeds unseen inputs and emits outputs with those weights frozen. The two modes require mutually exclusive training and test facts.

How facts are split

An 80/20 split that lifts every fifth chronological fact can put all Friday observations in the test subset and only Monday-Thursday observations in the training subset. Facts should be shuffled before the cut and again after it.

After the main split, a third mutually exclusive subset can hold the facts judged most important for judging quality. That reserved comparison set is used to compare competing trained networks.

Train and test subsets should keep similar distributions of the characteristics thought to matter, identified by statistical review or clustering. Sparsely represented outliers may be dropped before training and scoring.

Stepping across the error surface

If every initial weight is identical and the mapping needs unequal weights, learning can stall because error updates scale with the current weights. Small random initial weights are used instead.

On an error surface, an oversized learning rate can bounce from one side of a valley to the other, while an undersized rate delays descent. The search uses the largest learning rate that does not oscillate, and a momentum term can allow a larger rate without extra oscillation.

Simulated annealing uses a temperature-linked schedule. It starts with large steps so the search can move quickly across the error surface, then lowers the step size so later updates are less likely to oscillate while settling near a minimum.

Halt when test error rises

Overtraining is treated as a robustness failure in which the model memorizes training-set quirks and loses generalization on unseen facts. Training is paused at set intervals, the frozen network is scored on the test subset, and interim configurations that meet the error rules are saved. Rising test error is the signal to stop rather than to keep fitting the training subset.

Scoring should match the intended output. Mean distance-to-target measures fit a next-session high forecast. Directional or shape agreement fits movement. Procedure-level criteria fit a model that emits trade signals rather than price levels.