Brain functional connectivity has been demonstrated to serve as a "fingerprint" to predict individual behaviors and phenotypes. A precise mapping between them could provide insightful clues to brain architectures and the generation of cognition. In this context, the naturalistic paradigm provides more engaging conditions and richer fMRI information, and both preserves or even enhances individual features and increases sensitivity to phenotypic measures, compared with other functional MRI modalities including resting-state and task paradigms. However, to the best of our knowledge, only linear methods were developed for predicting phenotypic measures from brain activity under naturalistic stimulus, while the brain activity is highly dynamic and nonlinear. Hence, we adopted the nonlinear graph convolutional network (GCN) to predict cognition-related phenotypic score from brain functional connectivity under naturalistic stimulus, where subjects are the nodes and functional connectivity is node feature. The behavior patterns of eye movement were integrated into this method to estimate similarity across subjects and define the graph edges. A few nodes are labeled by their phenotypic score, and the model is trained to predict the scores of those unlabeled nodes. The prediction accuracy of this method outperforms those from the linear classification method, resting-state based functional node feature and random edge tests.
Over the past few years, Federated Learning (FL) has become an emerging machine learning technique to tackle data privacy challenges through collaborative training. In the Federated Learning algorithm, the clients submit a locally trained model, and the server aggregates these parameters until convergence. Despite significant efforts that have been made to FL in fields like computer vision, audio, and natural language processing, the FL applications utilizing multimodal data streams remain largely unexplored. It is known that multimodal learning has broad real-world applications in emotion recognition, healthcare, multimedia, and social media, while user privacy persists as a critical concern. Specifically, there are no existing FL benchmarks targeting multimodal applications or related tasks. In order to facilitate the research in multimodal FL, we introduce FedMultimodal, the first FL benchmark for multimodal learning covering five representative multimodal applications from ten commonly used datasets with a total of eight unique modalities. FedMultimodal offers a systematic FL pipeline, enabling end-to-end modeling framework ranging from data partition and feature extraction to FL benchmark algorithms and model evaluation. Unlike existing FL benchmarks, FedMultimodal provides a standardized approach to assess the robustness of FL against three common data corruptions in real-life multimodal applications: missing modalities, missing labels, and erroneous labels. We hope that FedMultimodal can accelerate numerous future research directions, including designing multimodal FL algorithms toward extreme data heterogeneity, robustness multimodal FL, and efficient multimodal FL. The datasets and benchmark results can be accessed at: https://github.com/usc-sail/fed-multimodal.
Graph neural networks (GNNs) have received increasing interest in the medical imaging field given their powerful graph embedding ability to characterize the non-Euclidean structure of brain networks based on magnetic resonance imaging (MRI) data. However, previous studies are largely node-centralized and ignore edge features for graph classification tasks, resulting in moderate performance of graph classification accuracy. Moreover, the generalizability of GNN model is still far from satisfactory in brain disorder [e.g., autism spectrum disorder (ASD)] identification due to considerable individual differences in symptoms among patients as well as data heterogeneity among different sites. In order to address the above limitations, this study proposes a novel adversarial learning-based node-edge graph attention network (AL-NEGAT) for ASD identification based on multimodal MRI data. First, both node and edge features are modeled based on structural and functional MRI data to leverage complementary brain information and preserved in the constructed weighted adjacent matrix for individuals through the attention mechanism in the proposed NEGAT. Second, two AL methods are employed to improve the generalizability of NEGAT. Finally, a gradient-based saliency map strategy is utilized for model interpretation to identify important brain regions and connections contributing to the classification. Experimental results based on the public Autism Brain Imaging Data Exchange I (ABIDE I) data demonstrate that the proposed framework achieves a classification accuracy of 74.7% between ASD and typical developing (TD) groups based on 1007 subjects across 17 different sites and outperforms the state-of-the-art methods, indicating satisfying classification ability and generalizability of the proposed AL-NEGAT model. Our work provides a powerful tool for brain disorder identification.
Abstract The important mechanical parameters and their hierarchy in the growth and folding of the human brain have not been thoroughly understood. In this study, we developed a multiscale mechanical model to investigate how the interplay between initial geometrical undulations, differential tangential growth in the cortical plate, and axonal connectivity form and regulate the folding patterns of the human brain in a hierarchical order. To do so, different growth scenarios with bilayer spherical models that features initial undulations on the cortex and uniform or heterogeneous distribution of axonal fibers in the white matter were developed, statistically analyzed, and validated by the imaging observations. The results showed that the differential tangential growth is the inducer of cortical folding, and in a hierarchal order, high-amplitude initial undulations on the surface and axonal fibers in the substrate regulate the folding patterns and determine the location of gyri and sulci. The locations with dense axonal fibers after folding settle in gyri rather than sulci. The statistical results also indicated that there is a strong correlation between the location of positive (outward) and negative (inward) initial undulations and the locations of gyri and sulci after folding, respectively. In addition, the locations of 3-hinge gyral folds are strongly correlated with the initial positive undulations and locations of dense axonal fibers. As another finding, it was revealed that there is a correlation between the density of axonal fibers and local gyrification index, which has been observed in imaging studies but not yet fundamentally explained. This study is the first step in understanding the linkage between abnormal gyrification (surface morphology) and disruption in connectivity that has been observed in some brain disorders such as Autism Spectrum Disorder. Moreover, the findings of the study directly contribute to the concept of the regularity and variability of folding patterns in individual human brains.
This paper firstly introduces the principle of neural network,and then proposes a method based on neural network to recognize the verification code. The method is implemented in Java using JOONE,which is an open-source neural network package. Finally,the experimental results of the traing samples show that the method has a higher recognition rate.
There is a significant relevance of federated learning (FL) in the realm of Artificial Intelligence of Things (AIoT). However, most existing FL works are not conducted on datasets collected from authentic IoT devices that capture unique modalities and inherent challenges of IoT data. In this work, we introduce FedAIoT, an FL benchmark for AIoT to fill this critical gap. FedAIoT includes eight datatsets collected from a wide range of IoT devices. These datasets cover unique IoT modalities and target representative applications of AIoT. FedAIoT also includes a unified end-to-end FL framework for AIoT that simplifies benchmarking the performance of the datasets. Our benchmark results shed light on the opportunities and challenges of FL for AIoT. We hope FedAIoT could serve as an invaluable resource to foster advancements in the important field of FL for AIoT. The repository of FedAIoT is maintained at https://github.com/AIoT-MLSys-Lab/FedAIoT.
Federated learning (FL) has gained substantial attention in recent years due to the data privacy concerns related to the pervasiveness of consumer devices that continuously collect data from users. While a number of FL benchmarks have been developed to facilitate FL research, none of them include audio data and audio-related tasks. In this paper, we fill this critical gap by introducing a new FL benchmark for audio tasks which we refer to as FedAudio. FedAudio includes four representative and commonly used audio datasets from three important audio tasks that are well aligned with FL use cases. In particular, a unique contribution of FedAudio is the introduction of data noises and label errors to the datasets to emulate challenges when deploying FL systems in real-world settings. FedAudio also includes the benchmark results of the datasets and a PyTorch library with the objective of facilitating researchers to fairly compare their algorithms. We hope FedAudio could act as a catalyst to inspire new FL research for audio tasks and thus benefit the acoustic and speech research community. The datasets and benchmark results can be accessed at https://github.com/zhang-tuo-pdf/FedAudio.
'/%3e%3cuse xlink:href='%23a' transform='rotate(23.036 .093 25.536)'/%3e%3cuse xlink:href='%23a' transform='rotate(45.87 1.273 16.18)'/%3e%3cuse xlink:href='%23a' transform='rotate(69.945 .996 12.078)'/%3e%3cuse xlink:href='%23a' transform='rotate(20.66 -19.689 31.932)'/%3e%3c/svg%3e)
