Weighted Fuzzy Dempster–Shafer Framework for Multimodal Information Integration

This study proposes an architecture based on a weighted fuzzy Dempster–Shafer framework (WFDSF), which can adjust weights associated with inconsistent evidence obtained by different classification approaches, to realize a fusion system for integrating multimodal information. The Dempster–Shafer theory (D-S theory) of evidence enables us to integrate heterogeneous information from multiple sources to obtain collaborative inferences for a given problem. To conquer various uncertainties associated with the collected information, our system assigns beliefs and plausibilities to possible hypotheses of each decision maker and uses a combination rule to fuse multimodal information. For information fusion, an important step in D-S aggregation is to find an appropriate basic probability assignment scheme for allocating support to each possible hypothesis/class, which remains an arduous and unsolved problem. Here, we propose a mathematical structure to aggregate weighted evidence extracted from two different types of approaches: fuzzy Naive Bayes and nearest mean classification rule. Further, an intuitionistic belief assignment is employed to address uncertainties between hypotheses/classes. Finally, 12 benchmark problems from the UCI machine learning repository for classification are employed to validate the proposed WFDSF-based scheme. In addition, an application of WFDSF to a practical brain–computer interface problem involving multimodal data fusion is demonstrated in this study. The experimental results show that the WFDSF is superior to several existing methods.