4. Computational Mechanisms of the Visual Processing of Action Stimuli

The recognition of body motion is a central function of the visual system that has stimulated substantial interest in neuroscience. At the same time, the recognition of body shapes, movements and actions from videos represents a complex computational problem, whose difficulty is sometimes bypassed by popular explanations of motion recognition in neuroscience. Only a serious interaction between neuroscience and computational theory will help to To appear in: K. Johnson & M. Shiffrar (Eds.) Perception of the Human Body in Motion: Findings, Theory and Practice. Oxford University Press. In press. 2 identify the important computational steps of action recognition in the brain, and might contribute a clarification of their neural implementation. Computational models can specifically help to test the computational feasibility of possible explanations of the processing of body movements, and they help to derive theoretically well-defined predictions that can be tested experimentally. Such theoretical work helps to derive critical constraints for explanations of the processing of body motion, since some intuitive theories might be computationally not feasible or not robust enough to deal with realworld stimuli. This chapter reviews a class of neural theories for the recognition of body motion, which was originally developed in order to account for the processing of biological motion stimuli and the recognition of non-transitive body movements, that is non goal-directed movements such as walking. We show how these theories can be extended to models for the processing of goal-directed transitive actions, that is actions with a goal object such as grasping. We show that such an extension is possible by addition of a few simple physiologically plausible neural mechanisms. The resulting model accounts for the view-independent recognition of hand actions from real videos with an accuracy that is sufficient even for the detection of subtle differences between grips. In addition, the resulting model reproduces a number of key properties of the visual tuning of action-selective neurons in visual, parietal and premotor cortex. The relationship between this new model and other computational approaches for the visual processing of goal-directed actions is discussed.