Density-based Multimodal Spatial Clustering using Pre-trained Deep Network for Extracting Local Topics

Users on social networking services (SNSs) have been transmitting information about events they witnessed themselves in their daily life through geo-social data as geo-tagged texts and photos. Geo-social data are usually related to not only personal topics but also local topics and events. Therefore, extracting local topics and events in geo-social data is one of the most important challenges in many application domains. In this study, to extract local topics in geo-social data, we propose a new method based on a density-based multimodal spatial clustering algorithm called the (e, σ)-density-based multimodal spatial clustering, which can extract multimodal spatial clusters that are spatially and semantically separated from other spatial clusters. Moreover, to present the main topics of each multimodal spatial cluster, representative photos are detected using network-based importance analysis. The proposed method utilizes a pre-trained deep network for extracting feature vectors of photos, and feature vectors are utilized to calculate the similarity between two geo-social data. To evaluate our new local topic extraction method, we conducted experiments using actual geo-tagged tweets that include photos. The experimental results show that the proposed method can extract local topics as multimodal spatial clusters more sensitively than our previous method.