Cascade error projection: a new learning algorithm

Artificial neural networks, with massive parallelism, have been shown to efficiently solve ill-defined problems in pattern recognition, classification, and optimization. It is further recognized that for real time low power operation, their highly parallel connectivity must be harnessed in VLSI silicon. However, such an implementation, specially an analog embodiment, suffers from two drawbacks: limited precision confounded by circuit noise and lack of proper hardware implementable learning algorithm. This paper specifically addresses these issues and offers an architecture along with a new hardware implementable learning algorithm called cascade error projection (CEP). CEP's methodology is tolerant of limited hardware precision, particularly the synapse resolution. A concise mathematical analysis is presented. The frame work can be used to obtain cascade correlation (CC) learning algorithm as a special case of CEP by choosing a particular set of parameters. Amongst its many attractive features one important CEP attribute is that it operates on one layer of input weights at a time while other connected weights are either frozen or deterministically calculable, hence provides fast learning. Finally, with limited synaptic weight resolution of only 3- to 4-bit, this technique is capable of learning reliably 5- to 8-bit parity problems by incorporating additional hidden units.