Error Backpropagation for Fully Connected Cascade Networks

This paper proposes an improvement of the BPFCC algorithm for computing the gradient for fully connected cascade (FCC) networks. BPFCC explicitly computes the gradient of the network function, and therefore for a multi-output $(q\gt 1)$ FCC network, q gradients need to be computed, which require q backpropagation passes. Realizing that it is the gradient of the error function that is truly necessary, we find that there is an elegant backpropagation procedure for directly computing the error gradient in one pass, thus dramatically reducing computation time for multi-output FCC networks. This algorithm, which is called EBPFCC, is rigorously derived and described in full detail in this paper, and its speed enhancement over the original BPFCC is demonstrated via numerical experiments.