Classification of focal liver lesions in CT images using convolutional neural networks with lesion information augmented patches and synthetic data augmentation.

PURPOSE We propose a deep learning method that classifies focal liver lesions (FLLs) into cysts, hemangiomas, and metastases from portal phase abdominal CT images. We propose a synthetic data augmentation process to alleviate the class imbalance and the Lesion INformation Augmented (LINA) patch to improve the learning efficiency. METHODS A dataset of 502 portal phase CT scans of 1,290 FLLs was used. First, to alleviate the class imbalance and to diversify the training data patterns, we suggest synthetic training data augmentation using DCGAN-based lesion mask synthesis and pix2pix-based mask-to-image translation. Second, to improve the learning efficiency of convolutional neural networks (CNNs) for the small lesions, we propose a novel type of input patch termed the LINA patch to emphasize the lesion texture information while also maintaining the lesion boundary information in the patches. Third, we construct a multi-scale CNN through a model ensemble of ResNet-18 CNNs trained on LINA patches of various mini-patch sizes. RESULTS The experiments demonstrate that (a) synthetic data augmentation method shows characteristics different but complementary to those in conventional real data augmentation in augmenting data distributions, (b) the proposed LINA patches improve classification performance compared to those by existing types of CNN input patches due to the enhanced texture and boundary information in the small lesions, and (c) through an ensemble of LINA patch-trained CNNs with different mini-patch sizes, the multi-scale CNN further improves overall classification performance. As a result, the proposed method achieved an accuracy of 87.30%, showing improvements of 10.81%p and 15.0%p compared to the conventional image patch-trained CNN and texture feature-trained SVM, respectively. CONCLUSIONS The proposed synthetic data augmentation method shows promising results in improving the data diversity and class imbalance, and the proposed LINA patches enhance the learning efficiency compared to the existing input image patches.