Deep learning in Nuclear Medicine—focus on CNN-based approaches for PET/CT and PET/MR: where do we stand?

In recent years, machine learning algorithms have led to innovative tools for medical imaging analysis. The purpose of the present review was to summarize the literature on the developing field of deep learning (DL), particularly the application of convolutional neural networks (CNNs) in PET/CT and PET/MR. We performed the literature search, referring to “convolutional neural networks” and “positron emission tomography” on PubMed/MEDLINE, for potentially relevant articles published up until July 24th, 2020. After the screening process, 63 articles were finally included; these embraced both the technical (n = 23) and the clinical field (n = 40). Technical studies aimed at investigating the role of CNN-based methods for image quality improvement (n = 11) and on technical issues (n = 12), mainly attenuation correction. Clinical studies explored CNN applications in oncology lung cancer (n = 7), head and neck cancer (n = 4), esophageal cancer (n = 2), lymphoma (n = 3), prostate cancer (N = 4), cervical cancer (n = 1), sarcomas (n = 1), multiple cancer types (n = 4), in neurology (n = 10) and cardiology (n = 1); three additional studies belonged to “other” category. In oncology, the studies aimed at detection, diagnosis, and prognostication of cancer. In neurology, the majority of the studies aimed at diagnosing Alzheimer Disease and stratification of the risk. CNN-based algorithms demonstrated promising results with performances equal or even higher compared to conventional approaches. Overall, CNN applications for PET/CT and PET/MR are exponentially growing, demonstrating encouraging results for both technical and clinical purposes. Novel research strategies emerged to face the challenges of DL algorithms development. Education and confidence with DL-based tools are needed for proper technology implementation.