Skip to main content
Track Robustness testing
6 / 51
Library

1989issue C111-8

Parameter neighborhoods that survive a shift

Parameter choice is framed as finding settings that stay profitable while limiting equity drawdown after an optimal-parameter-shift. Neighborhoods are kept when neighbors support the center and a 10% move in any parameter still passes a profit-to-drawdown-screen.

  • Treat parameter choice as finding settings that stay profitable and limit equity drawdown as conditions change, not as maximizing closed cumulative profit.
  • Aim for an operating point that can take an optimal-parameter-shift of 25% or more and still avoid an annual loss for any commodity under test.
  • Run a wide-scan and then a narrow-scan until a 10% move in any parameter still meets the annual 2.5-to-1 profit-to-drawdown-screen, and prefer the lower peak-to-trough-maximum-equity-drawdown when two neighborhoods score almost the same.
  • Keep a sector-central-value only when neighbors support the average-sector-value; discard an equity-spike and recompute the score after removing any single trade that accounts for more than 10% of closed profit.
Entries in this reading3 entries

Settings that stay profitable as conditions change

Parameter choice is framed as finding settings that stay profitable while limiting equity drawdown as conditions change, not as maximizing closed cumulative profit.

Optimal-parameter-shift is the change, from one sample window to the next, in the settings that look best on a completed-trade score. Commodity systems are said to be able to remain profitable for a given market after shifts as large as 50%. The stated search target is an operating point profitable enough that shifts of 25% or more still avoid an annual loss for any commodity under test.

How the completed-trade score is kept

The evaluation protocol uses one-contract trading, 50 to 100 currency units of commission and slippage per closed trade, no entries on locked-limit days, and at least six years of prices.

Open trades are dropped from the score unless they have already exceeded their peak equity. Intra-trade swings are accumulated as peak-to-trough-maximum-equity-drawdown until a new equity high, after which the accumulator resets.

Wide scan and narrow scan

Optimization is described as a wide-scan, then a narrow-scan, continued until a 10% move in any parameter still meets an annual 2.5-to-1 profit-to-drawdown-screen. If two neighborhoods score almost the same on that screen, the lower peak-to-trough-maximum-equity-drawdown is preferred.

Wide scan: closed profit by price-band and moving-average period

Closed cumulative profit peaks at $101,700 on a 25-day average with a 3.3% band, and nearby 20- and 30-day rows stay on the same ridge instead of collapsing. Those exact cells come from the article’s wide-scan table, so the neighborhood around the later 25-day / 3.3% choice is visible before any single richest cell is crowned.
Closed cumulative profit peaks at $101,700 on a 25-day average with a 3.3% band, and nearby 20- and 30-day rows stay on the same ridge instead of collapsing. Those exact cells come from the article’s wide-scan table, so the neighborhood around the later 25-day / 3.3% choice is visible before any single richest cell is crowned.T-bonds · daily

Profits are closed cumulative results in thousands of dollars after $100 commission and slippage per trade, as reported in the wide-scan grid.

Sector values and equity spikes

Average-sector-value is the mean of a nine-cell grid around a candidate center. The sector-central-value is to be used only when neighboring cells keep closed profit from falling sharply.

A candidate center is treated as an equity-spike and not used for trading if it sits 10% or more above any neighbor. A related check is whether more than 10% of closed profit comes from a single trade. If it does, that trade is removed and the score is recomputed.

The moving-average band example

In the moving-average-plus-band example, a close through the upper edge of a percentage-price-band is a buy, a close through the lower edge is a sell, and the moving average is the exit stop. A 10% change in lookback or band width is expected to change closed profit by no more than 10% if the center is robust.

The first-pass moving-average wide-scan covers lookbacks from 5 to 35 days and band widths from 2.4% to 4.5%. Later neighborhood comparisons of drawdown and the profit-to-drawdown-screen are used to pick among several candidate centers rather than taking the single largest closed-profit cell.

Educational research material, not investment advice. Historical source context does not establish present-day performance.
6 of 51 in the Robustness testing track
19901-7 pp.Next on Robustness testingUse profit mapping to keep a cycle and stop plateauSearching every combination of two rule parameters for the single historically highest profit is treated as fitting the model to back data and is not, by itself, evidence that those parameters will remain useful after market characteristics change.
All readings on this track · 51 readings
  1. 1986Degrees of freedom in trading system optimization
  2. 1988Walk-forward and neighborhood tests after optimization
  3. 1988Undisclosed rules block system robustness tests
  4. 1988Testing re-optimization calendars against random parameter controls
  5. 1989Binary search limits on multi-peak average grids
  6. 1989Parameter neighborhoods that survive a shift
  7. 1990Use profit mapping to keep a cycle and stop plateau
  8. 1990Why popular indicator optimization fails robustness
  9. 1991Retesting weighted indicator balances across horizons
  10. 1992Constructing forecast models with regression, walk-forward, and robustness
  11. 1992Diagnose regimes before you lock parameters
  12. 1992When stops change system timing
  13. 1993Walk-forward halt rules for forecast models
  14. 1994Walk-forward evaluation of genetic index rules
  15. 1995Input pruning as walk-forward system evaluation
  16. 1995Critiquing neural nets as incomplete trading systems
  17. 1996Rebuild the equity-path ratio before it ranks a designed system
  18. 1996Parameter grids can fit random walks
  19. 1996Walk-forward analysis belongs in the design of a mechanical trading system
  20. 1997When a holdout fails, discard the rule set
  21. 1997Test rewarded rule breaks before replacing the system
  22. 1997Walk-forward rules keep system research from rewriting live trades
  23. 1999Keep a channel-breakout to two lookbacks and test neighbor stability
  24. 1999Constant investment size in stock system evaluation
  25. 2000Forcing optimization maps mechanical system failure boundaries
  26. 2000Robust parameter selection with surface charts
  27. 2001A two-gate classroom test for a two-window momentum trend filter
  28. 2002How a two-sided continuation factor becomes a testable trend rule
  29. 2002Evaluating two-window trend intensity as a reversal rule
  30. 2003Discounting speculative bubbles in system robustness tests
  31. 2003Walk-forward evaluation of locked stochastic oscillator rules
  32. 2003Critiquing mechanical system design after extreme price regimes
  33. 2004Evaluating a two-window trend trigger
  34. 2005Grade backtested signals with holdouts and optimization plateaus
  35. 2006Reserved-sample evaluation of trading system design
  36. 2006Walk-forward critique of hindsight crossover systems
  37. 2008Condition-matched walk-forward evaluation for mechanical systems
  38. 2011Session-split evaluation of regular and overnight systems
  39. 2012Walk-forward evaluation as operator rehearsal
  40. 2013Two-window evaluation of mechanical trading systems
  41. 2013Walk-forward filter selection for repeated-median velocity
  42. 2014Walk-forward evaluation for fading-memory velocity systems
  43. 2015Test oscillator events before tuning rules
  44. 2016Walk-forward evaluation of a five-parameter parabolic stop-and-reversal
  45. 2016Walk-forward optimization without curve fitting
  46. 2017Optimization without overfitting in trend-system evaluation
  47. 2017Parameter stability is a better guide than a larger crossover grid
  48. 2018Point-in-time universes for system evaluation
  49. 2018Walk-forward robustness evaluation for optimized systems
  50. 2018Critiquing breakout systems through robustness tests
  51. 2018A critique of parameter fitting in system design
All 58 readings tagged Robustness testing
Also on Robustness testing5 readings