Constructing mechanical systems for walk-forward tests

The system is one testable procedure

A mechanical trading system is a fully specified procedure that turns rule inputs, market state and execution constraints into entry, exit and abstention signals over the system holding period, so the whole loop can be tested as one object.

In the archive workflow, a system is the joint of model, rules and parameters, and data series. The procedure is assembled by inspecting history, proposing rules and testing data that were not used to form those rules, so signals and quantified risk and opportunity estimates replace discretionary judgment.

Risk on hypothetical, backtested, walk-forward or live trades is checked by asking whether every signal could have been taken with available capital. Most such combinations cannot.

Choose the series before the rules

Price-series choice comes before rule search. Series that are too violent leave no safe mechanical procedure. Series that are too quiet lack usable opportunity.

Two- or three-times leveraged funds were rejected as too risky to host the system.

Both windows have to persist

Pattern persistence must cover both the in-sample construction window and a later out-of-sample window. In-sample data is the history used to discover patterns and to keep adjusting rules and parameters until the fit looks acceptable. Out-of-sample data is later history, preferably the same series and more recent, that was not used to form the rules and is the first check that the fit is not only noise.

Relatively short recent spans are preferred because stale or incorrect data is treated as worse than having no data.

Stationarity is stability of the series and of the system’s result distribution across the research window and the later test window. Fitting methods that assume stationary data are treated as failing once that stationarity ends. Folding position size into the signal model or simply trading through drawdowns both rest on that assumption.

Complexity and holding time

Extra rule complexity overfits the construction sample. A stated construction preference is frequent, accurate, short-hold trades because adverse excursion is described as growing with the square root of holding time. Stripping the worst losses from a trade set is treated as more important than keeping the best wins.

In-sample search is incomplete

Historical backtesting is required to see what was profitable on past data and is the usual in-sample search. Stopping development there supplies no estimate of later performance.

System optimization is that in-sample search among alternative rules and parameters, ranked by an objective-function score. It is incomplete until later unused data is applied. The objective function is the scoring rule that names which candidate is best at each in-sample step of a walk-forward run.

Walk-forward scoring

Walk-forward analysis is a stepped validation cycle that selects rules and parameters on an earlier window, tests them on a later unused window, then advances both windows through time. It repeatedly picks rules and parameters on in-sample data by an objective-function score, tests them on a later unused slice, then advances the pair by the out-of-sample length.

The concatenated unused-slice trades are used as the working estimate of later behavior. Each step is treated as practice for leaving development.

Write every path, then size the tape

After a system is chosen, every entry and every exit path must be written as a rule with no subjective exits. Impulse signals are events that open or close a position, such as buy, sell, short or cover. State signals name the next-bar stance, long, flat or short, including every bar between an entry impulse and the matching exit. With end-of-day mark-to-market, that stance is the intended position for the following day.

A second model takes the trade sequence as data and outputs position size. Position size is kept outside the signal model so live trade-sequence quality can still be observed. The size rule described here is anti-martingale sizing: it raises exposure after acceptable results and cuts exposure after poor results, including a zero-size halt when the profit estimate falls below other uses of the same funds.

In the archive workflow, among systems examined none supported a size above zero when holding exceeded two weeks.