A Multilingual Exploration of Semantics in the Brain Using Tensor Decomposition

The semantic concept processing mechanism of the brain shows that different neural activity patterns occur for different semantic categories. Multivariate Pattern Analysis of the brain fMRI data shows promising results in identifying active brain regions for a specific semantic category. Unsupervised learning technique such as tensor decomposition discovers the hidden structure from the brain data and proved to be useful as well. However, the existing methods are used for analyzing data from subjects who speak in one language and do not consider the cultural effect on it. This thesis presents an exploratory analysis of the neuro-semantic problem in a new dimension. The brain fMRI tensors of subjects who speak in Chinese or Italian language are analyzed both individually and together to discover the hidden structure. The Chinese and Italian tensors are jointly analyzed by coupling them along the stimuli object mode to discover the cultural effect. Moreover, the joint analysis of semantic features and brain fMRI tensor using the Advanced Coupled Matrix Tensor Factorization (ACMTF) method finds latent variables that explain the correlation between them. The results of the joint analysis of the tensors support the preliminary predictive analysis and find meaningful clusters for the different categories of stimuli object. Moreover, for a rank 2 decomposition, the prediction of brain activation pattern given semantic features gives an accuracy of 71.43%. It is expected that, the proposed exploratory and predictive analysis will improve existing approaches of analyzing conceptual knowledge representation of brain and guide future research in this domain.