Hierarchical Convolutional Neural Networks Information Fusion for Activity Source Detection in Smart Buildings

Detecting the source of activities in buildings is of interest to enable the potentials for Smart Buildings in industrial facilities, public infrastructure, hospitals, and commercial or residential buildings. This problem has been addressed using intrusive techniques, including vision-based methods. Recent studies have instead demonstrated that unobtrusive approaches could provide comparable results to the intrusive methods. These methods have the capability to identify and classify atypical floor vibration signals to specific sources, e.g., human fall detection or rotary machines fault detection. Most successful applications of vibration-based monitoring use traditional feature learning methods. These methods extract the temporal and spectral features from the vibration signals, and human experts are involved in selecting the features for classification. However, they typically require a labor-intensive process, which may add uncertainty and bias to the results. Several promising alternatives to these approaches have recently been proposed which take advantage of the time-dependent frequency spectrum of the vibration signals (i.e., spectogram). These representations are then fed into a deep convolutional neural network (CNN) architecture for image-like classification, training an accurate model. However, activty source detection in buildings requires the identification of numerous possible sources (classes), which may lead to low accuracy and confidence in the results. Thus, in this study, we incorporate prior knowledge of a target application, called a hierarchical structure of activity sources (classes). We develop a hierarchical CNN-based technique, which has a relatively easy setup, is modifiable, and has an extendable structure, to use floor vibration signatures to detect the source of activities in a building. We validate the method on a recently released benchmark dataset for human activity identification by implementing a flat numerous-class CNN-based approach, and the proposed hierarchical CNN-based approach. The results indicate that the hierarchical CNN-based approach outperforms other methods.