Artificial intelligence using convolutional neural networks for real-time detection of early esophageal neoplasia in Barrett’s esophagus (with video)

Abstract Background and Aims The visual detection of early esophageal neoplasia (high-grade dysplasia and T1 cancer) in Barrett’s esophagus (BE) with white-light and virtual chromoendoscopy still remains challenging. The aim of this study was to assess whether a convolutional neural artificial intelligence network can aid in the recognition of early esophageal neoplasia in BE. Methods Nine hundred sixteen images from 65 patients were collected of histology-proven early esophageal neoplasia in BE containing high-grade dysplasia or T1 cancer. The area of neoplasia was masked using image annotation software. Nine hundred nineteen control images were collected of BE without high-grade dysplasia. A convolutional neural network (CNN) algorithm was pretrained on ImageNet and then fine-tuned with the goal to provide the correct binary classification of “dysplastic” or “nondysplastic.” We developed an object detection algorithm that drew localization boxes around regions classified as dysplasia. Results The CNN analyzed 458 test images (225 dysplasia/233 nondysplasia) and correctly detected early neoplasia with sensitivity of 96.4%, specificity of 94.2% and accuracy of 95.4%. With regard to the object detection algorithm for all images in the validation set, the system was able to achieve a mean-average-precision (mAP) of 0.7533 at an intersection over union (IOU) of 0.3 Conclusion In this pilot study, our AI model was able to detect early esophageal neoplasia in Barrett’s esophagus images with high accuracy. In addition, the object detection algorithm was able to draw a localization box around the areas of dysplasia with high precision and at a speed that allows for real-time implementation.