Covariance Self-Attention Dual Path UNet for Rectal Tumor Segmentation

Deep learning algorithms are recognized as the most effective method for rectal tumor segmentation. However, since the multi-scale detailed feature information of rectal tumor cannot be fully extracted and applied, the segmentation and identification results of most algorithms are not always perfect. In this work, we introduce a Covariance Self-Attention Dual Path UNet (CSA-DPUNet), that is modified on the basis of UNet network and self-attention mechanism to improve network performance in feature processing and representation. The proposed network mainly makes two improvements. First, the UNet structure with single path is extended to dual paths (DPUNet). By broadening the network connections, our network is able to learn more local features with multiple contextual scales from CT images. Second, an improved criss-cross self-attention module is incorporated into DPUNet(CSA-DPUNet), instead of correlation method, we adopt covariance operation to calculate the attention weight map of self-attention mechanism, which can adaptively enhance feature combination and characterization ability. Experiments illustrate that our network called CSA-DPUNet can obviously improves the segmentation accuracy of rectal tumors, which brings 15.31%, 7.2%, 11.8%, and 9.5% improvement in Dice coefficient, P, R, F1, respectively compared with state-of-the-art. The above characteristics make the proposed CSA-DPUNet suitable for segmenting rectal tumor in practice.