A visualization method based on the Grad-CAM for medical image segmentation model

Computer-aided diagnosis technology based on the deep learning greatly increases the efficiency of medical diagnosis tasks. However, the black-box nature of deep neural networks reduces the reliability of auxiliary clinical diagnosis, so it is necessary to explain the medical imaging diagnosis model based on deep learning. The Gradient-weighted Class Activation Mapping (Grad-CAM) can determine the contribution of input features to the result of the classification task and visualize these contributions as heatmaps. However, due to the structural difference between the classification model and the segmentation model, Grad-CAM cannot be directly used for the deep neural network of medical image segmentation to generate interpretable heatmaps. Therefore, we propose an improved Grad-CAM visualization method. First, the output mask of the segmentation model is converted into a column vector, and the segmentation is performed by setting a threshold strategy. Then, to generate the visualization results of the medical image segmentation model, the sum of pixels exceeding a threshold is propagated in reverse, and then the contribution of input pixels to segmentation results is obtained. Finally, the proposed method is applied to the three medical image segmentation models, Double U-Net, R2U-Net, and MCGU-Net. And the effectiveness of the proposed method is verified in the three medical image datasets, thereby generating accurate interpretable heatmap. A large number of experiments demonstrated that the proposed method can be effectively applied to explain medical image segmentation models.