A Reinforcement Meta-Learning Framework of Executive Function and Information Demand

Gathering information is crucial for maximizing fitness, but requires diverting resources from searching directly for primary rewards to actively exploring the environment. Optimal decision-making thus maximizes information while reducing effort costs, but little is known about the neural implementation of these tradeoffs. We present a Reinforcement Meta-Learning (RML) computational mechanism that solves the trade-offs between the value and costs of gathering information. We implement the RML in a biologically plausible architecture that links catecholaminergic neuromodulators, the medial prefrontal cortex and topographically organized visual maps and show that it accounts for neural and behavioral findings on information demand motivated by instrumental incentives and intrinsic utility. Moreover, the utility function used by the RML, encoded by dopamine, is an approximation of free-energy. Thus, the RML presents a biologically plausible mechanism through which coordinated motivational, executive and sensory systems generate visual information gathering policies that minimize free energy.