Improving the malignancy characterization of hepatocellular carcinoma using deeply supervised cross modal transfer learning for non-enhanced MR

The malignancy characterization of hepatocellular carcinoma (HCC) is remarkably significant in clinical practice. In this work, we propose a deeply supervised cross modal transfer learning method to remarkably improve the malignancy characterization of HCC based on non-enhanced MR. First, we use samples of non-enhanced and contrast-enhanced MR for pre-training a deep learning network to learn the cross modal relationship between the non-enhanced modal and enhanced modal. Then, the parameters of the pre-trained across modal representation are transferred to a second deep learning model for fine-tuning based only on non-enhanced MR for malignancy characterization of HCC. Specifically, a deeply supervised network is designed to enhance the stability of the second deep learning model and further improve the performance of lesion characterization. Importantly, only non-enhanced MR of HCC is required for the malignancy characterization in the training and test phase of the second deep learning model. Experiments of one hundred and fifteen clinical HCCs demonstrate that the proposed deeply supervised cross modal transfer learning method can significantly improve the malignancy characterization of HCC based on non-enhanced MR.