Glomerular classification using convolutional neural networks based on defined annotation criteria and concordance evaluation among clinicians

Abstract Introduction Diagnosing renal pathologies is important for performing treatments. However, classifying every glomerulus is difficult for clinicians; thus, a support system, such as a computer, is required. This paper describes the automatic classification of glomerular images using a convolutional neural network (CNN). Method To generate appropriate labeled data, annotation criteria including 12 features e.g., “fibrous crescent”, were defined. The concordance among five clinicians was evaluated for 100 images using kappa coefficient for each feature. Using the annotation criteria, one clinician annotated 10,102 images. We trained the CNNs to classify the features whose average kappa ≧ 0.4 and evaluated their performance using the receiver operating characteristic – area under the curve (ROC–AUC). An error analysis was conducted and the gradient-weighted class activation mapping (Grad-CAM) was also applied; it expresses the CNN’s focusing point with a heat map when the CNN classifies the glomerular image for a feature. Result The average kappa coefficient of the features ranged from 0.28 to 0.50. The ROC–AUC of the CNNs for test data varied from 0.65 to 0.98. Among the features, “capillary collapse” and “fibrous crescent” had high ROC–AUC values of 0.98 and 0.91, respectively. The error analysis and the Grad-CAM visually showed the CNN could not distinguish between two different features that had similar visual structures or occurred simultaneously. Conclusion The differences in the texture or frequency of the co-occurrence between the different features affected the CNN performance; thus, to improve the classification accuracy, methods such as segmentation are required.