Skip to main content
Track Robustness testing
16 / 51
Search the archiveLibrary
Library

1995issue C121-5

Critiquing neural nets as incomplete trading systems

The archive workflow treats a neural net as one modeling choice inside a larger procedure. Editorial reading judges that procedure only after system-optimization, walk-forward-analysis, and robustness-testing can still score the same entry, exit, and abstention rules when the architecture is swapped.

  • Try many input and modeling approaches rather than hunt for one privileged neural-network recipe.
  • The usual reported breakthrough is a change in the relative-target, not a change in architecture or the input list.
  • input-engineering, a short training-window, a refined validation-set, and an ensemble-signal rank above network type.
  • Editorial reading: judge the design only after system-optimization, walk-forward-analysis, and robustness-testing can still score the same rules when the architecture is swapped.
Entries in this reading3 entries
Current trackRobustness testing58 readingsSystem optimization85 readingsWalk-forward analysis95 readings

A replaceable signal block

The described practice is to try many input and modeling approaches rather than hunt for one privileged neural-network recipe. Inputs and network structure are treated as objects of ongoing redesign because markets change and no compact fixed variable set is said to capture them.

Change what is predicted

The usual reported breakthrough is a change in what is predicted, not a change in architecture or the input list. Forecasts are framed as a relative-target, meaning price change versus an index or moving average rather than raw price, mainly to normalize series through time.

Next-day market forecasts are generally avoided because daily prices are described as noisier and more emotion-driven than longer horizons. Individual stocks are treated as easier targets than a broad index, and several predicted-strong and predicted-weak names are taken together to diversify model error.

Inputs and the training-window

Usable input sets are described as combining six to 30 features from fundamentals, technicals, and ratios. That input-engineering step is required because open, high, low, close, and volume alone are called insufficient.

The training-window is kept to roughly 200 to 2,000 patterns because long histories worsen normalization and mix shifted market conditions.

Holdouts and an ensemble-signal

A validation-set is run during training to limit memorization. Holdout construction is itself refined rather than taken as a pure random sample or only the latest bars. That is the role of walk-forward-analysis in this workflow.

Multiple networks are trained for the same decision and combined as an ensemble-signal by vote, average, or a second selector. Network type is ranked below data design among the factors that matter. The same procedure is then put through robustness-testing across issues, models, and regimes so one network or one name cannot carry the signal.

Score the same rules after a swap

system-optimization is the joint search over inputs, target definition, and execution constraints so entry, exit, and abstention rules can be scored as one procedure.

Editorial reading adds one check the archive workflow already makes room for. Keep the entry, exit, and abstention rules fixed, swap the architecture, and accept the design only if system-optimization, walk-forward-analysis, and robustness-testing can still score those rules.

Educational research material, not investment advice. Historical source context does not establish present-day performance.
16 of 51 in the Robustness testing track
19961-5 pp.Next on Robustness testingRebuild the equity-path ratio before it ranks a designed systemA published ratio that fits a line through successive equity observations was later rewritten after the original implementation misapplied the intended statistical formula.
All readings on this track · 51 readings
  1. 1986Degrees of freedom in trading system optimization
  2. 1988Walk-forward and neighborhood tests after optimization
  3. 1988Undisclosed rules block system robustness tests
  4. 1988Testing re-optimization calendars against random parameter controls
  5. 1989Binary search limits on multi-peak average grids
  6. 1989Parameter neighborhoods that survive a shift
  7. 1990Use profit mapping to keep a cycle and stop plateau
  8. 1990Why popular indicator optimization fails robustness
  9. 1991Retesting weighted indicator balances across horizons
  10. 1992Constructing forecast models with regression, walk-forward, and robustness
  11. 1992Diagnose regimes before you lock parameters
  12. 1992When stops change system timing
  13. 1993Walk-forward halt rules for forecast models
  14. 1994Walk-forward evaluation of genetic index rules
  15. 1995Input pruning as walk-forward system evaluation
  16. 1995Critiquing neural nets as incomplete trading systems
  17. 1996Rebuild the equity-path ratio before it ranks a designed system
  18. 1996Parameter grids can fit random walks
  19. 1996Walk-forward analysis belongs in the design of a mechanical trading system
  20. 1997When a holdout fails, discard the rule set
  21. 1997Test rewarded rule breaks before replacing the system
  22. 1997Walk-forward rules keep system research from rewriting live trades
  23. 1999Keep a channel-breakout to two lookbacks and test neighbor stability
  24. 1999Constant investment size in stock system evaluation
  25. 2000Forcing optimization maps mechanical system failure boundaries
  26. 2000Robust parameter selection with surface charts
  27. 2001A two-gate classroom test for a two-window momentum trend filter
  28. 2002How a two-sided continuation factor becomes a testable trend rule
  29. 2002Evaluating two-window trend intensity as a reversal rule
  30. 2003Discounting speculative bubbles in system robustness tests
  31. 2003Walk-forward evaluation of locked stochastic oscillator rules
  32. 2003Critiquing mechanical system design after extreme price regimes
  33. 2004Evaluating a two-window trend trigger
  34. 2005Grade backtested signals with holdouts and optimization plateaus
  35. 2006Reserved-sample evaluation of trading system design
  36. 2006Walk-forward critique of hindsight crossover systems
  37. 2008Condition-matched walk-forward evaluation for mechanical systems
  38. 2011Session-split evaluation of regular and overnight systems
  39. 2012Walk-forward evaluation as operator rehearsal
  40. 2013Two-window evaluation of mechanical trading systems
  41. 2013Walk-forward filter selection for repeated-median velocity
  42. 2014Walk-forward evaluation for fading-memory velocity systems
  43. 2015Test oscillator events before tuning rules
  44. 2016Walk-forward evaluation of a five-parameter parabolic stop-and-reversal
  45. 2016Walk-forward optimization without curve fitting
  46. 2017Optimization without overfitting in trend-system evaluation
  47. 2017Parameter stability is a better guide than a larger crossover grid
  48. 2018Point-in-time universes for system evaluation
  49. 2018Walk-forward robustness evaluation for optimized systems
  50. 2018Critiquing breakout systems through robustness tests
  51. 2018A critique of parameter fitting in system design
By fieldSystem design117 readingsBy year199575 readings
All 58 readings tagged Robustness testing
Also on Robustness testing5 readings
1986Degrees of freedom in trading system optimization1988Testing re-optimization calendars against random parameter controls1988Walk-forward and neighborhood tests after optimization1992Diagnose regimes before you lock parameters1993Walk-forward halt rules for forecast models