Using Multimodal Learning Model for Earthquake Damage Detection Based on Optical Satellite Imagery and Structural Attributes

Herein, we propose a novel scheme for detecting earthquake-damaged buildings from optical satellite imageries. The scheme comprises two steps: first, using the information of the positions and shapes of the buildings in a GIS database, we identify the photographic scope of each residence in wide-area photographic imagery and extract small photographic fragments at the level of individual structures. Second, using a classifier to determine if these individual fragments represent collapsed structures, we assess the damage of the residential structures in the affected area. Furthermore, we verify the effectiveness of applying the recent machine learning techniques to improve the performance of the classifier. To utilize images captured before and immediately after the earthquake, we apply spatio-temporal convolution neural network that is regarded as a generalized method of image subtraction. Additionally, we integrate the structural attributes comprising structural age and structural materials with the satellite images using a multimodal learning structure. The effectiveness of using the aforementioned techniques is discussed using a dataset constructed from a satellite imagery of the affected area in 2016 Kumamoto Earthquake Japan taken by Spot 6 and 7.