A GRU Neural Network with attention mechanism for detection of risk situations on multimodal lifelog data

Multimedia today is also in multimodality. Working with heterogeneous signals we use multimedia techniques of data fusion and mining. Classification from real world datasets are often challenging. The paper is devoted to the detection of personal risk situations of fragile people from multi-modal sensing real world lifelog data named BIRDS. Using a real-world data is challenging as the risk situations are rare and last just a few seconds compared to the global volume of the dataset. In this paper we propose a GRU architecture with global attention block to recognise semantic risk situations from a limited taxonomy. Attention is also focused on data organisation and pre-processing with imputation and normalisation. The proposed method is applied to a real-world collected multimodal dataset and to the OpenSource dataset UCI-HAR for the sake of comparison with the state-of-the-art.