2013issue C0852
Constructing a null and a t-test for sample returns
Lock a confidence level before any test is run, write a null that two procedures cannot be told apart, then form a t-style statistic from the sample mean, a stated baseline mean, and dispersion scaled by the observation count.
- A confidence level has to be selected before statistical tests are run, because that choice decides when the null hypothesis is accepted.
- Under the five-percent rule, the null is not accepted when a sample average net return would appear in only five of one hundred similar samples if the null were true.
- The null hypothesis is the claim that a specified procedure cannot validly distinguish two variations of the same thing.
- A t-test checks a distribution whose shape changes as the observation count shrinks, using the gap between the sample mean and the population mean divided by the sample standard deviation scaled by the square root of the observation count.
A build order
Editorial reading: assemble the comparison in this order. Lock the confidence level first. Write a null that two procedures are indistinguishable. Then assemble a t-style statistic from the sample mean, a stated baseline mean, and dispersion scaled by the observation count.
Lock the confidence level
A confidence level has to be selected before statistical tests are run, because that choice is what decides when the null hypothesis is accepted.
The five-percent rule is one such preselected rule. Under that rule, the null is not accepted when a sample average net return would arise in only five of one hundred similar samples if the null were true.
Write the null
The null hypothesis is the claim that a specified procedure cannot validly distinguish two variations of the same thing.
Editorial reading: write that claim as the working statement that the two procedures under comparison cannot validly be told apart.
Assemble the t-statistic
A t-test is a significance test for a distribution whose shape changes as the observation count becomes smaller.
The t-statistic is the gap between the sample mean and the population mean, divided by the sample standard deviation scaled by the square root of the number of observations.
Editorial reading: treat the population mean as the stated baseline mean of the comparison, and treat the divisor as dispersion scaled by sample size.
Read the dispersion term
Standard deviation describes how a distribution varies around its mean observation.
On a profit-and-loss series, a large standard deviation indicates widely varying outcomes and a small one indicates more tightly clustered outcomes.
All readings on this track · 20 readings
- 1988Serial dependence, trade excursions, and belief revision
- 1989Score an indicator before anyone writes a buy or sell rule
- 1990Evaluating safety labels in a crash window
- 1991Keep a decade-year only when it differs from the overall-mean
- 1991Audit indicator forecasts with chi-square hit rates
- 1992A lag-regression audit of stochastic forecast skill
- 1992Constructing detrended tests for calendar seasonality
- 1992Building a shared workbook for calendar seasonality tests
- 1995T-test evaluation of simulated trade returns
- 1995Constructing a two-proportion Z-test for hit rates
- 1996Treat a volume-price surge as a contingency table
- 1997Testing bond and stock seasonality before you trade
- 1997Walk-forward tests after system optimization
- 1997Testing breakout rate versus rally speed
- 2000Score the misses before a pattern is a system
- 2002Leave a moving-average crossover unfinished until chance cannot explain the ranges
- 2013Keep or drop an RSI threshold with a complementary-sample test
- 2013Constructing a null and a t-test for sample returns
- 2017Partial correlation and the market-adjusted stock trend
- 2017Constructing market-adjusted stock trend scores