Critiquing mechanical system design after extreme price regimes

The procedure is the unit of the test

A mechanical trading system is a fully specified procedure whose output is a signal from rule inputs, market state, and execution constraints over the system holding period, so entry, exit, and abstention can be tested as one procedure. System optimization adjusts those rule inputs until the backtested signal fits a chosen historical path. The fit is useful only if the same procedure remains testable outside that path.

Extreme multi-year paths make it easier to overfit a mechanical system so that the fitted rules describe a unique historical episode rather than a repeatable signal. After a large multi-year rise and collapse, that risk shows up at once on the chart, in the ranking, and in the calendar of signals.

Chart scale after a rise and collapse

Robustness testing checks that the same signal still appears, and remains executable, when chart scale is changed. An arithmetic price scale spaces equal currency increments equally and can hide large percentage moves after price has already collapsed. After a large multi-year rise and collapse, that scale can make a 12.5 percent drop near a former high look larger than a 50 percent drop near the new low.

A logarithmic price scale spaces equal percentage changes equally so long, high-to-low histories can be compared on one chart. Equal percentage changes sit at equal vertical distances, so the span from 10 to 20 matches 20 to 40 and 40 to 80. A long post-collapse path that looks like consolidation on an arithmetic chart can read as a continued downtrend.

Ranking math after cheap names remain

Relative-strength ranking is a sort of names by comparative percentage movement that can elevate low-priced rebounds after a broad decline. Rankings built from percentage change systematically favor low-priced names. After a broad collapse many widely followed names sit below a former 10 price cutoff, so excluding cheap names no longer clears the ranking.

Percentage-average bias is the tendency of averaged successive percentage changes to look positive after a round trip that returns price to its start. A fall of 50 percent followed by a rise of 100 percent averages positive even though price is unchanged. A one-year relative-strength recipe that splits history into quarters and double-weights the latest quarter can rank a recently rebounded low-priced name at the top even when that name remains below its level of a year earlier.

When almost every signal falls in one year

A purchase rule that requires a large short-window rise in a major-index moving average can concentrate almost every historical signal in one boom year and then produce none through later declining years. That pattern is a regime-bound signal: a backtested trigger that fires densely in one unusual window and then goes silent, so the historical sample is not a later opportunity set.

When nearly all signals arrive in a single year, a book that is already fully invested cannot take the later same-year signals, so a trade-level backtest overstates what capital rules would have allowed. A mechanical model fitted to a severe multi-year decline is not shown to remain useful if the following years are less severe, just as a model fitted to a boom year is not shown to remain useful after that boom ends.

Three robustness tests that fail together

Editorial view: chart scale, ranking math, and the calendar of signals are three robustness tests that fail together after an extreme regime. Changing only one surface can leave the mechanical trading system looking intact while the other two still describe a unique historical episode.

Leave the procedure in place only after robustness testing has changed chart scale, ranking math, capital constraints, and the historical regime, and the same signal still appears and remains executable.