Personality Prediction with Cross-Modality Feature Projection

In this paper, we propose an approach to predict customers’ personalities leveraging two modalities of customers’ data: (a) service usages logs of online services and (b) visual data collected in physical stores by surveillance cameras (e.g., gait and whereabouts). A number of companies provide services via online and offline nowadays, thus need to serve two different kinds of customers: “online customers,” who use services completely online and have only (a); and “offline customers,” who use only physical stores and have only (b). To improve personality prediction accuracy for these customers, our approach generates pseudo features of a non-existent modality from the other modality that the customers actually have (i.e., feature projection; e.g., generate pseudo visual data of the online customers from their real online service logs), and uses both pseudo and real features to predict their personalities. The evaluation using real-world data of a mobile carrier’s customers showed that our approach predicted personality more accurately than an ordinary unimodal approach for both online and offline customers. We also examined importance of the feature projection and compared two different projection methods.