Pre-Consultation System Based on the Artificial Intelligence Has a Better Diagnostic Performance Than the Physicians in the Outpatient Department of Pediatrics

Artificial intelligence has been deeply applied in the medical field and has shown broad application prospects. Pre-consultation is an important supplement to traditional face-to-face consultation. The combination of artificial intelligence (AI) and the pre-consultation system can help raise the efficiency of clinical work. However, it is still challenging for AI to analyze and process the complicated electronic health record data (EHR). Our pre-consultation system uses an automated natural language processing (NLP) system to communicate with patients through mobile terminals, applying deep learning (DL) techniques to extract symptomatic information and finally outputs structured electronic medical records. From November, 2019 to May, 2020, a total of 2648 pediatric patients used our model to provide their medical history and get primary diagnosis before visiting physicians in the outpatient department of Shanghai Children’s Medical Center. Our task is to evaluate the ability of AI and doctors to obtain primary diagnosis, and to analyze the effect of consistency between medical history described by our model and the physicians on the diagnostic performance. The results showed that if we do not consider whether medical history recorded by AI and doctors were consistent or not, our model performed worse than physicians and had a lower average F1 score (0.825 vs. 0.912). However, when the chief complaint or history of present illness described by artificial intelligence and doctors were consistent, our model had a higher average F1 score and closer to doctors’. Finally, when the AI had the same diagnostic conditions with doctors, our model achieved a higher average F1 score (0.931) than physicians (0.92). Our research demonstrated that our model could obtain more structured medical history and had a good diagnostic logic, which would help improve the diagnostic accuracy of outpatient doctors and reduce misdiagnosis and missed diagnosis. But our model still needs a good deal of training to obtain more accurate symptomatic information.