Multi-granularity scale-aware networks for hard pixels segmentation of pulmonary nodules

Abstract Accurate automatic segmentation of pulmonary nodules can greatly assist in the early clinical diagnosis and analysis of lung cancer. However, it remains a challenging task due to (1) the variable scales, complex shapes, and textures heterogeneity of nodules and (2) the similar visual characteristics of nodules and the surrounding environment. These factors make it difficult for a model to capture the representative and distinguishing features of nodules. We define these pixels, which are difficult to segment accurately, as “hard pixels.” To comprehensively tackle these challenges, we focus on the complementarity between surrounding background information and salient nodules information. Accordingly, we propose a Multi-Granularity Scale-Aware Networks (MGSA-Net) for accurate pulmonary nodules segmentation with steps as follows. First, to effectively preserve both global contextual and local fine details information, we unify the representation of the feature about global and patch-level images in a single framework. Second, we introduce a deep scale-aware module (DSAM) in the global stream that could generate multi-scale feature maps with a uniform representational power, which can process more contextual information. Finally, we propose a multi-granularity feature map sharing learning to fuse feature maps from the dual branch at various scales. Benefiting from the rich background information and salient nodules information, the fused features can help simultaneously capture the similarity of nodules and the diversity of background to mine hard pixels. Especially, the two streams are jointly optimized, ensuring they are mutually reinforced and refining current pixels’ predictions by their similar structure boundaries. Extensive experiments on the LIDC-IDRI dataset have demonstrated that the proposed MGSA-Net could surpass most segmentation models and advance the state-of-the-art performance with the dice similarity coefficient (DSC) of 87.32% on the dataset. Code can be available at: https://github.com/ISSE-AILab/MGSA-Net.