Audit mechanical system tests for fills and regimes

Grade the test before the rule

Editorial: before asking whether a mechanical moving-average rule worked, ask which unstated gifts were doing the work. Zero friction, illegal or same-bar fills, ignored crossings inside the bar, and a one-way sample can carry a historical score. Grade the honesty of the simulator before grading the procedure.

A mechanical trading system is a complete rule procedure that issues specific buy and sell signals. A moving-average crossover is given as the elementary case. Composite mechanical systems may stack multiple moving averages with oscillators, stochastics, relative-strength, MACD, filters, and stop-loss rules. Historical testing is treated as applicable to both simple and composite stacks.

Historical price databases can score a mechanical strategy in minutes. A favorable historical result is described as failing to transfer when the test used incorrect assumptions.

Transaction costs can reverse the score

Omitting commissions and slippage, or setting them unrealistically low, is named a primary testing error. Transaction costs are commissions plus slippage charged against each completed trade inside a historical test.

In a 30-trade illustration, a 6,000 pre-cost total becomes 3,000, zero, or a 3,000 loss at per-trade costs of 100, 200, and 300.

Fill price must be later and increment-legal

The fill price is the price at which a test assumes a signal could actually have been executed after the rule fired. A crossover fill at 411.03 is rejected because the contract traded in 0.05 increments and because the buy decision is known only after the average has already been crossed. A later increment-legal price is required.

A close-triggered signal cannot be filled at that close. The next open is the realistic price. In a 63-session March to May 1992 index-futures sample, the open matched the prior close once, differed by 0.73 on average, and gapped 2.55 and nearly 3 points on two cited days.

Close-to-close tests can hide intraperiod trades

A daily close-to-close moving-average test can log one buy on a day whose intra-day path produced three buys and two sells. An intraperiod trade is an extra entry or exit created by the path inside a sampling bar and invisible to a close-to-close comparison. Ignored intraperiod trades overstate the procedure.

Regime coverage is part of the sample

Trend-following mechanical systems can look strong in a one-direction rise or decline and generate large losses in a sideways stretch. A sufficient sample must include rising, falling, and sideways movement. Regime coverage is the inclusion of those three stretches in the historical window used to judge a trend-following rule.

When a historical test is described as informative

Historical tests are described as informative only when cost, fill, path, and sample-period assumptions resemble live trading conditions.

Editorial: do not ask whether the moving-average crossover worked until the historical test has been audited for transaction costs, fill price, intraperiod trades, and regime coverage.