Reinforcement-guided learning in frontal neocortex: emerging computational concepts

The classical concepts of reinforcement learning in the mammalian brain focus on dopamine release in the basal ganglia as the neural substrate of reward prediction errors, which drive plasticity in striatal and cortico-striatal synapses to maximize the expected aggregate future reward. This temporal difference framework, however, even when augmented with deep credit assignment, does not fully capture higher-order processes such as the influence of goal representations, planning based on learned internal models, and hierarchical decision-making implemented by diverse neocortical areas. Candidate functions for such neocortical contributions to reinforcement learning are increasingly being considered in artificial intelligence algorithms. Here, we review recent experimental neurophysiological findings focusing on the orbitofrontal cortex, a key higher-order association cortex, and highlight emerging concepts that emphasize the role of the neocortex in reward-driven computation, in addition to its role as an input to striatal structures. In this framework, reward drives plasticity in various neocortical regions, implementing multiple distinct reinforcement learning algorithms.