Content-Aware Multi-task Neural Networks for User Gender Inference Based on Social Media Images

To estimate demographic attributes such as gender and age of social media users from images posted by the users is a challenging problem because the demographic attributes are directly not shown in images. For such problem, prior approaches can be roughly separated into two types: one approach uses concept detection to detect pre-defined visual concepts which are then used as meta-data to estimate demographic attributes and the other approach directly uses content features such as Fisher Vector [19] which are extracted from images. In this paper we consider the way of combining these two approaches. We propose Multi-task Bilinear Model for integrating the detected concepts with the content features. In our proposed method, both the concept detector and the feature extractor can be jointly learned with end-to-end fashion. We evaluated the proposed method for the task of estimating user gender from Twitter images and found that it outperformed other baseline methods.