Infrared and visible image fusion (IVIF) aims to fully preserve the target and detail information of the infrared and visible images in the fusion image. Although deep learning–based methods have been widely used in IVIF, they usually use the same network structure to extract features without considering the differences between different image modalities, leading to insufficient feature extraction and unsatisfactory fusion results. To overcome these problems, we propose a multiscale attention and cross-convolution network (MACCNet) to obtain competitive fusion results. The method includes a new two-branched structure-based encoder network for extracting features from two different modality images. In one branch, a new multiscale attention module (MAM) extracts the target features at different scales of the input infrared images. In the other branch, the new cross-convolution feature extraction module (CFEM) extracts the detail features of visible images in different directions. We also introduce a local saliency attention fusion network (LSAFN) to obtain two weight maps to improve the fusion of extracted target and detail features of the different modality images. Additionally, the two weight maps are averaged as coefficients of the pixel loss terms to adaptively train the network. Finally, we obtain the final fusion result by reconstructing the fused features via a decoder network. Experimental results show that the proposed MACCNet outperforms several state-of-the-art IVIF methods in terms of visual perception and objective evaluation.
For multi-sensor fusion of infrared and visible images, it is difficult to retain the thermal radiation information of the infrared image and the texture information of the visible image in the fused image. To overcome this problem, a novel infrared and visible image fusion method based on a modified side window filter (MSWF) and an intensity transformation is proposed. First, an MSWF with effective edge-preservation ability is developed by adding four additional kernels to better decompose the source images to obtain the base and detail layers. Furthermore, to extract the edge information of the source images, we propose to further decompose the base layers to obtain the low-frequency layers and high-frequency layers (edge information) through the non-subsampled shearlet transform (NSST). Then, an S-shape intensity transformation function (ITF) is proposed to enhance the saliency information and suppress the non-saliency information in the infrared image. In the fusion process, considering the characteristics of the decomposed components, different fusion rules are designed to obtain the fused detail layer and low- and high-frequency layers. Finally, these fused components are reconstructed to obtain the final fusion image. It is experimentally demonstrated that the proposed method is superior to state-of-the-art fusion methods both in terms of subjective evaluation and objective metrics.
Abstract Although deep neural network technology brings high recognition accuracy to the field of synthetic aperture radar image‐based automatic target recognition, it also produces the catastrophic forgetting problem. Here, a new incremental learning method that can extract more information about old data is proposed. Based on the rehearsal method, the authors’ method adds extra linear layers after the feature extractor of the network before training on new incremental data and uses the network to generate distilled labels for incremental training. Through experiments on the moving and stationary target acquisition and recognition data set, we conclude that, when the old model has good performance, our method has better performance than other typical incremental learning methods on small data sets.
In this paper, we present a new regularization-based approach to construct a high-resolution image. The objective functional of the approach consists of a data fidelity term and a regularization term based on anisotropic fourth-order diffusion (AFOD) prior. To evaluate and prove the performance of the proposed method, series of experiments and comparisons with some existing methods including bi-cubic interpolation method and bilateral total variation method are carried out in the paper. Numerical results based on qualitative and quantitative evaluation can indicate that our algorithm is effective.
Attention deficit hyperactivity disorder (ADHD) is the most common childhood neuropsychiatric disorder and affects approximately 6-7% of children worldwide. Here, we investigate the statistical properties of undirected and directed brain functional networks in ADHD patients based on random matrix theory (RMT), in which the undirected functional connectivity is constructed based on correlation coefficient and the directed functional connectivity is measured based on cross-correlation coefficient and mutual information. We first analyze the functional connectivity and the eigenvalues of the brain functional network. We find that ADHD patients have increased undirected functional connectivity, reflecting a higher degree of linear dependence between regions, and increased directed functional connectivity, indicating stronger causality and more transmission of information among brain regions. More importantly, we explore the randomness of the undirected and directed functional networks using RMT. We find that for ADHD patients, the undirected functional network is more orderly than that for normal subjects, which indicates an abnormal increase in undirected functional connectivity. In addition, we find that the directed functional networks are more random, which reveals greater disorder in causality and more chaotic information flow among brain regions in ADHD patients. Our results not only further confirm the efficacy of RMT in characterizing the intrinsic properties of brain functional networks but also provide insights into the possibilities RMT offers for improving clinical diagnoses and treatment evaluations for ADHD patients.
Being different from the most methods of guided depth map enhancement based on deep convolutional neural network which focus on increasing the depth of networks, this paper is to improve the effectiveness of intensity guidance when the network goes deep. Overall, the proposed network upsamples the low-resolution depth maps from coarse to fine. Within each refinement stage of certain-scale depth features, the current-scale and all coarse-scales of the guidance features are revisited by dense connection. Therefore, the multi-scale guidance is efficiently maintained as the propagation of features. Furthermore, the proposed network maintains the intensity features in the high-resolution domain from which the multi-scale guidance is directly extracted. This design further improves the quality of intensity guidance. In addition, the shallow depth features upsampled via transposed convolution layer are directly transferred to the final depth features for reconstruction, which is called global residual learning in feature domain. Similarly, the global residual learning in pixel domain learns the difference between the depth ground truth and the coarsely upsampled depth map. Also, the local residual learning is to maintain the low frequency within each refinement stage and progressively recover the high frequency. The proposed method is tested for noise-free and noisy cases which compares against 16 state-of-the-art methods. Our experimental results show the improved performances based on the qualitative and quantitative evaluations.
A new machine learning method referred to as F-score_ELM was proposed to classify the lying and truth-telling using the electroencephalogram (EEG) signals from 28 guilty and innocent subjects. Thirty-one features were extracted from the probe responses from these subjects. Then, a recently-developed classifier called extreme learning machine (ELM) was combined with F-score, a simple but effective feature selection method, to jointly optimize the number of the hidden nodes of ELM and the feature subset by a grid-searching training procedure. The method was compared to two classification models combining principal component analysis with back-propagation network and support vector machine classifiers. We thoroughly assessed the performance of these classification models including the training and testing time, sensitivity and specificity from the training and testing sets, as well as network size. The experimental results showed that the number of the hidden nodes can be effectively optimized by the proposed method. Also, F-score_ELM obtained the best classification accuracy and required the shortest training and testing time.
Previous studies have shown escitalopram is related to sleep quality. However, effects of escitalopram on dynamics of electroencephalogram (EEG) features especially during different sleep stages have not been reported. This study may help to reveal pharmacological mechanism underlying escitalopram treatment.The spatial and temporal responses of patients with major depressive disorder (MDD) to escitalopram treatment were analyzed in this study. Eleven MDD patients and eleven healthy control subjects who completed eight weeks' treatment of escitalopram were included in the final statistics. Six-channel sleep EEG signals were acquired during sleep. Power spectrum and nonlinear dynamics were used to analyze the spatio-temporal dynamics features of the sleep EEG after escitalopram treatment.For temporal dynamics: after treatment, there was a significant increase in the relative energy (RE) of δ1 band (0.5 - 2 Hz), accompanied by a significant decrease in the RE of β2 band (20 - 30 Hz). Lempel-Ziv complexity and Co - complexity values were significantly lower. EEG changes at different sleep stages also showed the same regulation as throughout the night sleep. For spatio dynamics: after treatment, the EEG response of the left and right hemisphere showed asymmetry. Regarding band-specific EEG complexity estimations, δ1 and β2 in stage-1 and δ1 in stage-2 sleep stage in frontal cortex is found to be much more sensitive to escitalopram treatment in comparison to central and occipital cortices.The sleep quality of MDD patients improved, EEG response occurred asymmetry in left and right hemispheres due to escitalopram treatment, and frontal cortex is found to be much more sensitive to escitalopram treatment. These findings may contribute to a comprehensive understanding of the pharmacological mechanism of escitalopram in the treatment of depression.
Hyperspectral (HS) pansharpening aims to fuse high-spatial-resolution panchromatic (PAN) images with low-spatial-resolution hyperspectral (LRHS) images to generate high-spatial-resolution hyperspectral (HRHS) images. Due to the lack of consideration for the modal feature difference between PAN and LRHS images, most deep leaning-based methods suffer from spectral and spatial distortions in the fusion results. In addition, most methods use upsampled LRHS images as network input, resulting in spectral distortion. To address these issues, we propose a dual-stage feature correction fusion network (DFCFN) that achieves accurate fusion of PAN and LRHS images by constructing two fusion sub-networks: a feature correction compensation fusion network (FCCFN) and a multi-scale spectral correction fusion network (MSCFN). Based on the lattice filter structure, FCCFN is designed to obtain the initial fusion result by mutually correcting and supplementing the modal features from PAN and LRHS images. To suppress spectral distortion and obtain fine HRHS results, MSCFN based on 2D discrete wavelet transform (2D-DWT) is constructed to gradually correct the spectral features of the initial fusion result by designing a conditional entropy transformer (CE-Transformer). Extensive experiments on three widely used simulated datasets and one real dataset demonstrate that the proposed DFCFN achieves significant improvements in both spatial and spectral quality metrics over other state-of-the-art (SOTA) methods. Specifically, the proposed method improves the SAM metric by 6.4%, 6.2%, and 5.3% compared to the second-best comparison approach on Pavia center, Botswana, and Chikusei datasets, respectively. The codes are made available at: https://github.com/EchoPhD/DFCFN.
'/%3e%3cuse xlink:href='%23a' transform='rotate(23.036 .093 25.536)'/%3e%3cuse xlink:href='%23a' transform='rotate(45.87 1.273 16.18)'/%3e%3cuse xlink:href='%23a' transform='rotate(69.945 .996 12.078)'/%3e%3cuse xlink:href='%23a' transform='rotate(20.66 -19.689 31.932)'/%3e%3c/svg%3e)