LDA-Enhanced Federated Learning for Image Classification with Missing Modality

In practical pattern recognition, e.g., image classification or recognition, the problem of missing modality, i.e., new patterns never trained by a learner pop up, can cause a dramatic decrease on the recognition accuracy. Existing algorithms as few-shot learning (FSL) and zero-shot learning (ZSL) have not sufficiently used information from other users or clients. If patterns or knowledge from other sources can be utilized as much as possible, the damage of missing modality is expected to be reduced. Privacy protection must be considered when trying to fetch information from other users. Motivated by these, an enhanced federated learning with linear discriminant analysis (LDA) is developed here. The data of each user is regarded as a client, and the features of each client are first extracted with neural network-based classification. These features are uploaded to the central server and then aggregated with LDA as a central classification to possibly achieve all patterns’ features. The trained LDA is then downloaded to the client to fulfill the pattern recognition. The proposed algorithm is applied to an image classification, and the experimental results demonstrate its efficiency in dealing with pattern recognition with missing modality.