On-line Learning of Prototypes and Principal Components

We review our recent investigation of on-line unsupervised learning from high-dimensional structured data. First, on-line competitive learning is studied as a method for the identification of prototype vectors from overlapping clusters of examples. Specifically, we analyse the dynamics of the well-known winner-takes-all or K-means algorithm. As a second standard learning technique, the application of Sanger's rule for principal component analysis is investigated. In both scenarios the necessary process of student specialization may be delayed significantly due to underlying symmetries.