online bootstrapping of sensory representations

This is a simulation-based contribution exploring a novel approach to the open-ended formation of multimodal representations in autonomous agents. In particular, we address the issue of transferring ("bootstrapping") feature selectivities between two modalities, from a previously learned or innate ref- erence representation to a new induced representation. We demonstrate the potential of this algorithm by several experiments with synthetic inputs mod- eled after a robotics scenario where multimodal object representations are "bootstrapped" from a (reference) representation of object aordances. We focus on typical challenges in autonomous agents: absence of human super- vision, changing environment statistics and limited computing power. We propose an autonomous and local neural learning algorithm termed PRO- PRE (projection-prediction) that updates induced representations based on predictability: competitive advantages are given to those feature-sensitive ele- ments that are inferable from activities in the reference representation. PRO- PRE implements a bi-directional interaction of clustering ("projection") and inference ("prediction"), the key ingredient being an ecient online measure of predictability controlling learning in the projection step. We show that the proposed method is computationally ecient and stable, and that th e multi- modal transfer of feature selectivity is successful and robust under resource constraints. Furthermore, we successfully demonstrate robustness to noisy reference representations, non-stationary input statistics and uninformative inputs.