The Role of the Basal Ganglia in Exploratory Behavior in a Model Based on Reinforcement Learning

We present a model of basal ganglia as a key player in exploratory behavior. The model describes exploration of a virtual rat in a simulated “water pool” experiment. The virtual rat is trained using a reward-based or reinforcement learning paradigm which requires units with stochastic behavior for exploration of the system’s state space. We model the STN-GPe system as a pair of neuronal layers with oscillatory dynamics, exhibiting a variety of dynamic regimes like chaos, traveling waves and clustering. Invoking the property of chaotic systems to explore a state space, we suggest that the complex “exploratory” dynamics of STN-GPe system in conjunction with dopamine-based reward signaling present the two key ingredients of a reinforcement learning system.