A Guided Attention 4D Convolutional Neural Network for Modeling Spatio-Temporal Patterns of Functional Brain Networks

Since the complex brain functions are achieved by the interaction of functional brain networks with the specific spatial distributions and temporal dynamics, modeling the spatial and temporal patterns of functional brain networks based on 4D fMRI data offers a way to understand the brain functional mechanisms. Matrix decomposition methods and deep learning methods have been developed to provide solutions. However, the underlying nature of functional brain networks remains unclear due to underutilizing the spatio-temporal characteristics of 4D fMRI input in previous methods. To address this problem, we propose a novel Guided Attention 4D Convolutional Neural Network (GA-4DCNN) to model spatial and temporal patterns of functional brain networks simultaneously. GA-4DCNN consists of two subnetworks: the spatial 4DCNN and the temporal Guided Attention (GA) network. The 4DCNN firstly extracts spatio-temporal characteristics of fMRI input to model the spatial pattern, while the GA network further models the corresponding temporal pattern guided by the modeled spatial pattern. Based on two task fMRI datasets from the Human Connectome Project, experimental results show that the proposed GA-4DCNN has superior ability and generalizability in modeling spatial and temporal patterns compared to other state-of-the-art methods. This study provides a new useful tool for modeling and understanding brain function.