Score an indicator before anyone writes a buy or sell rule

Rules score the rule, not only the indicator

Judging an indicator only through buy and sell rules tests the chosen rule as much as the indicator. That habit is the signal-event method: invent the rules first, then score the rules. The verdict attaches to the rule set as much as to the indicator, and more complex multi-condition rules raise the chance of a lucky historical fit.

Compare cell forecasts with the sample mean

The cells method sorts ordered indicator readings into ranges so that similar values share a bin, then asks whether each bin's later market change is more predictable than the sample-wide average. Grouping similar readings into cells yields a zero-to-100 variance-reduction score of how much a later percentage change is associated with the current reading.

Variance reduction measures how much a cell-conditioned forecast reduces error relative to a naive forecast that always uses the grand sample average. The naive forecast is the baseline prediction that every case will match the sample-wide average of the chosen future percentage change. The conditional forecast is the prediction taken from the average outcome inside the cell that matches the current indicator reading.

Variance reduction is positive only when cell-conditioned forecasts have smaller error than the grand-average naive forecast. Otherwise the measure is zero or negative. Near-perfect single-indicator scores are treated as unavailable in practice. Typical variance-reduction scores fall between zero and 10 percent, with higher scores described as requiring several complementary indicators.

A worked breadth-ratio example

The worked example paired a moving-average-smoothed advance-minus-decline ratio with a 60-day forward percentage change in a broad equity index. It used every other observation from a January 1945 to July 1988 sample of 10,708 readings and 10 equally populated cells.

Across 5,354 used cases the sample-mean 60-day change was +1.85 percent, while the highest-indicator cell of 535 cases averaged +3.22 percent. That cell mean is a conditional forecast judged by variance reduction against the sample mean.

A holdout split and a chance threshold

Positive variance reduction can appear by chance, especially when many indicators are screened, so both a learning-versus-test split and a significance threshold are required.

Cross-validation forms cell averages on a learning set and requires the same predictive relationship to appear on an independent testing set. That split of ordered price, volume, or breadth observations uses the learning set to form cell averages and the testing set to score those averages out of sample.

Hypothesis testing adds a formal check that an indicator's apparent forecast improvement is unlikely under a no-information null before the indicator is treated as informative. The chi-square test is the chance-rate gate on variance reduction. The threshold is set so a useless indicator would exceed it by chance only 5 percent of the time.

Ranking by horizon without a trade rule

After those checks, the illustrated breadth-ratio series showed little predictive information across five other smoothings and multiple forecast horizons. The same workflow still allowed indicators to be ranked by horizon without defining trade rules.