Neural-net construction as a mechanical trading-system problem

A mapping from inputs to an output

The archive presents a neural network as software that maps independent inputs to a dependent output. The learned relationship may be linear or nonlinear. It is taken from representative samples and later applied to new inputs.

This installment is the first of two parts. It covers network definition, optimization methods, and software design. Buy and sell rules, data preparation, and a method for verifying trading-system performance are held for the sequel.

How a node computes its result

Each network node sums incoming signals plus an optional bias. It then applies a nonlinear transfer function, such as a sigmoid, a Gaussian, or a sine, before passing the result onward. The described networks end at a single output node.

Training as system optimization

Training is framed as an optimization search. The search reduces the difference between desired and actual output by changing connection strengths and other internal characteristics. The archive compares that search to least-squares curve fitting.

Four optimization procedures are distinguished. Exhaustive enumeration tries every candidate. Stochastic search keeps random guesses that improve the result. Gradient descent follows the error slope. Genetic algorithms combine crossover with mutation.

Gradient descent is said to reach a global minimum quickly when the error surface is smoothly concave. Local minima can still trap the search away from that minimum.

Stochastic weights and automatic architecture

The software used stochastic weight search instead of gradient descent. It kept randomly chosen weights that reduced output error, shrank the sampling range as error fell, and allowed transfer functions with zero slope or discontinuous jumps.

Network architecture, including layer count, node count, transfer-function mix, and bias use, was selected automatically. Unattended genetic-algorithm trials tested a small number of internal layers and mixed transfer functions. A complexity penalty was used to limit overfitting.

Checks before market use

Before market use, the software was checked on exclusive-or, circle area, trigonometric and polynomial maps, sunspot and Canadian lynx series, and a large input matrix in which only a few columns determined the output.

What this part leaves for the sequel

The sequel is reserved for data preprocessing, probability-based buy and sell rules, and a method for verifying trading-system performance. Editorial: that later verification step is where Walk-forward analysis would complete the single Mechanical trading system procedure. This part does not supply those rules or that verification.