Skip to main content
Track Autocorrelation test
6 / 15
Search the archiveLibrary
Library

1990issue C041

Two-gate out-of-sample tests for trading rules

Editorial interpretation: walk a full entry, exit, and abstention rule set through unseen periods after a heavy cost overlay, then keep a series only if its quarterly out-of-sample mean clears zero and an autocorrelation test finds no memory of the prior quarter.

  • Walk-forward analysis scores entry, exit, and abstention rules together on unseen periods, so the whole procedure is tested rather than a single fitted coefficient.
  • A cost overlay of brokerage, slippage, and margin-financing charges is applied before any significance test, and that combined treatment was characterized as an unrealistically high cost burden.
  • A one-tailed t-test asks whether mean quarterly out-of-sample net returns exceed zero, after a Kolmogorov-Smirnov check that did not reject normality.
  • First-order autocorrelation of quarterly net returns was used to drop series that showed a positive mean but also significant positive serial correlation, which would break serial independence.
Entries in this reading2 entries
Current trackAutocorrelation test23 readingsWalk-forward analysis119 readings

A two-gate reading of the workflow

Editorial interpretation: treat evaluation as a two-gate procedure. First walk a full entry, exit, and abstention rule set through unseen periods after a deliberately heavy cost overlay. Then keep a series only if its out-of-sample quarterly mean clears zero and its residuals do not remember the prior quarter.

The points that follow describe the historical workflow. They are not a present-day market claim.

Walk the full rule set on unseen periods

Walk-forward analysis scores entry, exit, and abstention rules together on unseen periods so the whole procedure, not a single fitted coefficient, is what gets tested.

Fill timing was rule-specific. Some signals were assumed to trade at a computed price in the same session. Others were assumed to trade only on the next session open.

Any still-open contract was closed at the first session close of the expiration month.

Apply the cost overlay before any test

A cost overlay applies brokerage, slippage, and margin-financing charges to each series before any significance test.

Quarterly results used one contract after a $47.50 net friction charge, described as a $10 brokerage fee plus three ticks of slippage.

Those results were also reduced by the interest cost of a $3,500 margin at the matching 90-day Treasury bill rate, with no interest credited on equity above exchange minimum margin.

The combined friction and financing treatment was characterized as an unrealistically high cost burden.

Ask whether the quarterly mean exceeds zero

Mean quarterly out-of-sample net returns were tested against zero with a one-tailed t-test that assumed normality and serial independence.

A one-tailed t-test asks whether mean quarterly out-of-sample net returns exceed zero, rather than whether they merely differ from zero.

A Kolmogorov-Smirnov check screens the quarterly net-return sample for compatibility with normality before trusting the t-test. That check did not reject normality of the quarterly net-return samples used in the t-test.

Under that test, 20 of 34 combined long-and-short series, 7 short-only series, and 2 long-only series were counted as having means statistically above zero.

Drop series that remember the prior quarter

Serial independence is the requirement that successive quarterly net returns not carry memory of the previous quarter.

An autocorrelation test measures whether ordered out-of-sample returns still depend on their own lag, which would break the independence assumption behind a mean test.

First-order autocorrelation of quarterly net returns was used to drop series that otherwise showed a positive mean but also showed significant positive serial correlation.

Indicators with a significant positive quarterly mean

A trader should see that the first gate—mean quarterly out-of-sample net return greater than zero on a one-tailed t-test—was cleared much more often by combined short-plus-long books than by one-sided rules. The sidebar states the counts in prose: 20 of 34 indicators on short-plus-long positions, seven on exclusively short positions, and two on exclusively long positions. These are first-gate results only; the same paragraph notes that several of those winners later failed a first-order autocorrelation screen.
A trader should see that the first gate—mean quarterly out-of-sample net return greater than zero on a one-tailed t-test—was cleared much more often by combined short-plus-long books than by one-sided rules. The sidebar states the counts in prose: 20 of 34 indicators on short-plus-long positions, seven on exclusively short positions, and two on exclusively long positions. These are first-gate results only; the same paragraph notes that several of those winners later failed a first-order autocorrelation screen.quarterly out-of-sample net returns

Net returns already subtract $47.50 per contract ($10 brokerage plus three ticks of slippage) and interest on a $3,500 margin at the 90-day Treasury bill rate, which the authors call an unrealistically high cost overlay. The t-test assumes normality and serial independence; a Kolmogorov-Smirnov check did not reject normality of quarterly net returns.

Educational research material, not investment advice. Historical source context does not establish present-day performance.
6 of 15 in the Autocorrelation test track
19901-2 pp.Next on Autocorrelation testSign-cell correlation, chi-square checks, and lag shiftsTwo indicator series can be compared by converting each reading into a signed change and placing each contemporaneous pair into a four-cell sign table.
All readings on this track · 15 readings
  1. 1987Evaluating weekly price cycles with autocorrelation cutoffs
  2. 1988Serial correlation is not an independence test
  3. 1988Event-triggered chi-square tests for jittered serial links
  4. 1988Reverse the rule before you trust the system
  5. 1990Chi-square evaluation of lagged breadth correlations
  6. 1990Two-gate out-of-sample tests for trading rules
  7. 1990Sign-cell correlation, chi-square checks, and lag shifts
  8. 1998Test serial dependence before you optimize a mechanical system
  9. 2003How a one-rank rotation is constructed for an offset autocorrelation check
  10. 2004Constructing first-order residual autocorrelation tests
  11. 2004A two-stage Durbin-Watson residual screen
  12. 2011Constructing trend labels from increment correlation atoms
  13. 2012Overnight versus daytime return evaluation
  14. 2012Constructing a multi-horizon folded-correlation reversal filter
  15. 2016Tuning indicators to measured market cycles
By fieldStatistical tests95 readingsBy year1990142 readings
All 23 readings tagged Autocorrelation test
Also on Autocorrelation test5 readings
1982Construct ARIMA forecasts from lag diagnostics1986Evaluating daily price and volume serial independence windows1986Weekly resamples as a diagnostic filter for statistical windows1987Evaluating money-supply serial dependence before a forecast1987Evaluating weekly price cycles with autocorrelation cutoffs