Recognizing Human-Object Interactions via Target Localization

The recognition of human-object interactions is a challenging problem due to the variety of object appearance, body poses, occlusions and the scene layout. The difficulty is particularly pronounced in actions interacting with small and partially occluded objects. Indeed, it is difficult to identify those objects by general object detectors, which makes it hard for accurate recognition of human-object interactions. In order to deal with this challenge, we propose a target prediction model that aims to identify regions relevant to the human-object interactions. Our model predicts the precise target location relating to the specific action by formulating it to a fully convolutional network that enables fine-grained localization. We jointly learn the appearance and location of the target by exploiting the target-specific segmentation information. We show that our target prediction model outperforms state-of-the-art methods in identifying small and occluded objects, and its result can be used to improve the recognition of human-object interactions.