|
||||||
|
The Cascade Correlation AlgorithmDeveloped in 1990 by Fahlman, the cascade correlation algorithm provides a solution. Cascade correlation neural networks are similar to traditional networks in that the neuron is the most basic unit. Training the neurons, however, is rather novel. An untrained cascade correlation network is a blank slate; it has no hidden units. A cascade correlation network’s output weights are trained until either the solution is found, or progress stagnates. If a single layered network will suffice, training is complete.
The size step problem contributes greatly to error stagnation. When a network takes steps that are too small, the error value reaches an asymptote. Cascade correlation avoids this by using the quick-prop algorithm. Quick-prop finds the second derivative (change of slope) of the error function. With each epoch it takes a different size step corresponding to the change of the error slope. If the error is reducing rapidly, quick-prop descends quickly. Once the slope flattens out, quick-prop slows down so that it does not pass the minimum. Quick-prop is like a person searching for a house. If the person knows he is far away, he drives fast. Once he approaches the house, he slows down and looks more carefully
![]() The difference in training speeds is unmistakably clear from an analysis of back-propagation’s error graph versus that of cascade-correlation. Back-propagation slowly converges on a solution as the moving target problem and size-step problem strike. In some trials the hidden neurons become locked on the same feature, and the network never attains a solution. As visible in the graph, cascade correlation faces this same problem except that when the error reduction stagnates, cascade correlation recruits a new hidden neuron that will promote further error reduction. Cascade correlation aggressively seeks to reduce error at the fastest possible rate.
|