Differential diagnosis for esophageal protruded lesions using a deep convolution neural network in endoscopic images

Abstracts Background and Aims Recent advances in deep convolutional neural network (CNN) have proven remarkable results in digestive endoscopy. In this study, we aimed to develop CNN-based methodologies models for differential diagnosis of benign esophageal protruded lesions using endoscopic images acquired during real clinical settings. Methods We retrospectively collected 1217 patients who underwent the white-light endoscopy (WLI) and endoscopic ultrasonography (EUS) between January 2015 and April 2020. Three deep CNN models were developed to accomplish the following tasks: (1) identification of esophageal benign lesions from healthy controls using WLI images; (2) differentiation 3 subtypes of esophageal protruded lesions (including EL: esophageal leiomyoma, EC: esophageal cyst, and EP: esophageal papilloma) using WLE images; and (3) discrimination between EL and EC using EUS images . Six endoscopists blinded to patients’ clinical status were enrolled to interpret all images independently. Their diagnostic performance were evaluated and compared with the CNN models using area under receiver operating characteristic curve (AUC). Results For task 1, the CNN model achieved the AUC of 0.751 (95% CI, 0.652 - 0.850) in identifying benign esophageal lesions. For task 2, the proposed model using WLI images for esophageal protruded lesions differentiation achieved the AUC of 0.907 (95% CI, 0.835 - 0.979), 0.897 (95% CI, 0.841 - 0.953) and 0.868 (95% CI, 0.769 - 0.968) for EP, EL, and EC, respectively. The CNN model achieved equivalent or higher identification accuracy of EL and EC compared with skilled endoscopists. In the task of discrimination of EL from EC (task 3), the proposed CNN model had the AUC values of 0.739 (EL, 95% CI, 0.600 - 0.878) and 0.724 (EC, 95% CI, 0.567 - 0.881), that outperformed seniors and novices. In attempts of combining CNN and endoscopists predictions lead to significantly improved diagnostic accuracy compared with endoscopists interpretations alone. Conclusions Our team managed to establish CNN-based methodologies to recognize benign esophageal protruded lesions using routinely obtained WLI and EUC images. Preliminary results that combining models and endoscopists results underscored the potential of ensemble models for improved lesions differentiation in real endoscopic settings.