AbstractThe basic idea of linear principal component analysis (PCA) involves decorrelating coordinates by an orthogonal linear transformation. In this paper we generalize this idea to the nonlinear case. Simultaneously we shall drop the usual restriction to Gaussian distributions. The linearity and orthogonality condition of linear PCA is replaced by the condition of volume conservation in order to avoid spurious information generated by the nonlinear transformation. This leads us to another very general class of nonlinear transformations, called symplectic maps. Later, instead of minimizing the correlation, we minimize the redundancy measured at the output coordinates. This generalizes second-order statistics, being only valid for Gaussian output distributions, to higher-order statistics. The proposed paradigm implements Barlow's redundancy-reduction principle for unsupervised feature extraction. The resulting factorial representation of the joint probability distribution presumably facilitates density estimation and is applied in particular to novelty detection.
AbstractThe basic idea of linear principal component analysis (PCA) involves decorrelating coordinates by an orthogonal linear transformation. In this paper we generalize this idea to the nonlinear case. Simultaneously we shall drop the usual restriction to Gaussian distributions. The linearity and orthogonality condition of linear PCA is replaced by the condition of volume conservation in order to avoid spurious information generated by the nonlinear transformation. This leads us to another very general class of nonlinear transformations, called symplectic maps. Later, instead of minimizing the correlation, we minimize the redundancy measured at the output coordinates. This generalizes second-order statistics, being only valid for Gaussian output distributions, to higher-order statistics. The proposed paradigm implements Barlow's redundancy-reduction principle for unsupervised feature extraction. The resulting factorial representation of the joint probability distribution presumably facilitates density estimation and is applied in particular to novelty detection.
According to Barlow (1989), feature extraction can be understood as finding a statistically independent representation of the probability distribution underlying the measured signals. The search for a statistically independent representation can be formulated by the criterion of minimal mutual information, which reduces to decorrelation in the case of gaussian distributions. If nongaussian distributions are to be considered, minimal mutual information is the appropriate generalization of decorrelation as used in linear Principal Component Analyses (PCA). We also generalize to nonlinear transformations by only demanding perfect transmission of information. This leads to a general class of nonlinear transformations, namely symplectic maps. Conservation of information allows us to consider only the statistics of single coordinates. The resulting factorial representation of the joint probability distribution gives a density estimation. We apply this concept to the real world problem of electrical motor fault detection treated as a novelty detection task.
The author provides calculus-of-variations techniques for the construction of backpropagation-through-time (BTT) algorithms for arbitrary time-dependent recurrent neural networks with both continuous and discrete dynamics. The backpropagated error signals are essentially Lagrange multipliers. The techniques are easy to handle because they can be embedded into the Hamiltonian formalism widely used in optimal control theory. Three examples of important extensions to the standard BTT-algorithm provide proof of the power of the method. An implementation of the BTT-algorithms which overcomes the storage drawbacks is suggested.< >
The paper deals with the numerical approximation of optimal strategies for two-person zero-sum differential games of pursuit-evasion type by neural networks. Thereby, the feedback strategies can be computed in real-time after the training of appropriate neural networks. For this purpose, sufficiently many optimal trajectories and their associated open-loop representations of the optimal feedback strategies must be computed, to provide data for training and cross-validation of the neural networks. All the precomputations can be carried through in a highly parallel way. This approach turns out to be applicable for differential games of more general type.The method is demonstrated for a modified cornered rat game where a pursuing cat and an evading rat, both moving in simple motion, are constrained to a rectangular arena. Two holes in the walls surrounding the arena enable the rat to evade once and for all, if the rat is not too far from these holes. The optimal trajectories in the escape zone can be computed analytically. In the capture zone, a game of degree is employed with terminal time as payoff. To compute optimal trajectories for this secondary game, the time evolution of the survival region for the rat is determined via a sequence of discretized games.The combination of these methods permits the computation of more than a thousand trajectories leading to some ten thousand sample patterns which relate the state variables to the values of the optimal strategies. These data exhibit characteristic properties of the optimal strategies. It is shown that these properties can be extracted from the data by use of neural networks. By means of the trained networks, about 200 trajectories arc finally simulated. The pursuer as well as the evader acts according to the controls proposed by the neural networks. Despite the simple structure of the neural networks used in this study, the strategics based upon them show a reasonable, close to optimal performance in a large variety of simulations of the pursuit-evasion game under consideration.
