Skip to main content
Track Robustness testing
25 / 51
Library

2000issue C021-5

Forcing optimization maps mechanical system failure boundaries

A mechanical system can produce a smooth, rising equity line on one historical window and then lose money on a later unused window under the same rules. Forcing tests rewrite slope or volatility so the designer can keep only the market-state envelope in which those rules still fire correctly.

  • The same moving-average and rate-of-change rules can produce a smooth, rising equity line on one historical window and then lose money on a later unused window.
  • Forcing optimization describes the mechanical system, designs tests that push it toward failure, and then interprets those failure conditions.
  • Forcing tests change slope or volatility while leaving other price features intact, so the evaluator can see which market-state changes the system cannot interpret.
  • Editorial reading: keep only the market-state envelope in which the same rules still fire correctly, and treat a speed measure such as standardized linear increase as a live filter when conditions leave that envelope.
Entries in this reading3 entries

A smooth equity line can still be sample-specific

A mechanical system that combines a moving average with a short and a long rate of change can produce a smooth, rising equity path on one historical window. The same rules and parameters can then lose money on a later unused window.

Walk-forward analysis applies those fixed rules and parameters to the unused period. The equity line is useful for spotting smooth in-sample results that later collapse on that later window.

Coffee system equity and price, 1976–81 backtest

The upper equity curve climbs from near zero to about $200,000 while coffee futures trend and oscillate below, matching the article’s claim that this 1977–81 window looks robust. Values were read from the MetaStock plot in Figure 1 (embedded-002), not from a numeric table.
The upper equity curve climbs from near zero to about $200,000 while coffee futures trend and oscillate below, matching the article’s claim that this 1977–81 window looks robust. Values were read from the MetaStock plot in Figure 1 (embedded-002), not from a numeric table.KC coffee futures · daily · 1976-01-01T00:00:00.000Z to 1981-12-31T00:00:00.000Z

Raster readout of the published MetaStock chart; equity and coffee price are approximate, sampled at yearly ticks. System parameters stated in the article: 30-day average, 15- and 25-day momentum, 0.5% commissions, 100% margin.

Forcing optimization is a three-step evaluation

Forcing optimization first states how a mechanical system should behave, then rewrites historical prices toward extreme slope or fluctuation, and finally maps the conditions under which the same rules stop working. In practice that means describe the mechanical system, design tests that push it toward failure, and then interpret those failure conditions.

A mechanical trading system is a fully specified set of entry, exit and reverse conditions driven by indicator inputs. The illustrated long rule requires price above an x-day moving average and both a y-day and a z-day rate of change above zero. The short rule is the exact reverse of those three conditions. The same procedure can be run on any price series.

Why a slower advance breaks the same rules

A moving average depends on the rate of price increase and on fluctuations around the trend. A slower advance can pull the average toward price so that a previously nearby fluctuation now crosses it. The same slowdown can shrink rate-of-change readings until a small drawdown turns them negative.

Forcing tests isolate one market feature

Forcing tests rewrite the original price series by changing slope or volatility while leaving other features intact. The evaluator can then see which market-state changes the system cannot interpret.

Selecting a backtest segment with an extreme slope and then widening its fluctuations, without changing other features, can show that a system with a good original equity path does not work in every market condition. Those tests also make the failure boundary identifiable. Robustness testing asks whether the same mechanical procedure still produces usable signals after the market features the rules depend on have been pushed beyond the original backtest sample.

Slope and volatility tests are not universal

Forcing tests built around slope and volatility are not automatically relevant to a relative-strength system. Different systems need different forcing tests aimed at their own limits of effectiveness.

A speed envelope can become a live filter

A standardized linear increase measures how fast price advances over a fixed lookback. It can mark a market that has left the tested speed envelope. The same mechanical system that worked above that speed did not work below it.

Editorial reading for system designers

Editorial reading: a profitable backtest remains an untested hypothesis until the designer deliberately warps slope and volatility. After those forcing tests, keep only the market-state envelope in which the same rules still fire correctly. Outside that failure boundary, a standardized linear increase can serve as a live filter for whether current conditions remain inside the tested envelope.

Educational research material, not investment advice. Historical source context does not establish present-day performance.
25 of 51 in the Robustness testing track
20001-6 pp.Next on Robustness testingRobust parameter selection with surface chartsLong-history tests, walk-forward holdouts, and visual inspection of a three-dimensional profit surface are complementary ways to judge whether parameter sets stay useful when conditions shift.
All readings on this track · 51 readings
  1. 1986Degrees of freedom in trading system optimization
  2. 1988Walk-forward and neighborhood tests after optimization
  3. 1988Undisclosed rules block system robustness tests
  4. 1988Testing re-optimization calendars against random parameter controls
  5. 1989Binary search limits on multi-peak average grids
  6. 1989Parameter neighborhoods that survive a shift
  7. 1990Use profit mapping to keep a cycle and stop plateau
  8. 1990Why popular indicator optimization fails robustness
  9. 1991Retesting weighted indicator balances across horizons
  10. 1992Constructing forecast models with regression, walk-forward, and robustness
  11. 1992Diagnose regimes before you lock parameters
  12. 1992When stops change system timing
  13. 1993Walk-forward halt rules for forecast models
  14. 1994Walk-forward evaluation of genetic index rules
  15. 1995Input pruning as walk-forward system evaluation
  16. 1995Critiquing neural nets as incomplete trading systems
  17. 1996Rebuild the equity-path ratio before it ranks a designed system
  18. 1996Parameter grids can fit random walks
  19. 1996Walk-forward analysis belongs in the design of a mechanical trading system
  20. 1997When a holdout fails, discard the rule set
  21. 1997Test rewarded rule breaks before replacing the system
  22. 1997Walk-forward rules keep system research from rewriting live trades
  23. 1999Keep a channel-breakout to two lookbacks and test neighbor stability
  24. 1999Constant investment size in stock system evaluation
  25. 2000Forcing optimization maps mechanical system failure boundaries
  26. 2000Robust parameter selection with surface charts
  27. 2001A two-gate classroom test for a two-window momentum trend filter
  28. 2002How a two-sided continuation factor becomes a testable trend rule
  29. 2002Evaluating two-window trend intensity as a reversal rule
  30. 2003Discounting speculative bubbles in system robustness tests
  31. 2003Walk-forward evaluation of locked stochastic oscillator rules
  32. 2003Critiquing mechanical system design after extreme price regimes
  33. 2004Evaluating a two-window trend trigger
  34. 2005Grade backtested signals with holdouts and optimization plateaus
  35. 2006Reserved-sample evaluation of trading system design
  36. 2006Walk-forward critique of hindsight crossover systems
  37. 2008Condition-matched walk-forward evaluation for mechanical systems
  38. 2011Session-split evaluation of regular and overnight systems
  39. 2012Walk-forward evaluation as operator rehearsal
  40. 2013Two-window evaluation of mechanical trading systems
  41. 2013Walk-forward filter selection for repeated-median velocity
  42. 2014Walk-forward evaluation for fading-memory velocity systems
  43. 2015Test oscillator events before tuning rules
  44. 2016Walk-forward evaluation of a five-parameter parabolic stop-and-reversal
  45. 2016Walk-forward optimization without curve fitting
  46. 2017Optimization without overfitting in trend-system evaluation
  47. 2017Parameter stability is a better guide than a larger crossover grid
  48. 2018Point-in-time universes for system evaluation
  49. 2018Walk-forward robustness evaluation for optimized systems
  50. 2018Critiquing breakout systems through robustness tests
  51. 2018A critique of parameter fitting in system design
All 58 readings tagged Robustness testing
Also on Robustness testing5 readings