Extending 2D Convolutional Neural Networks to 3D for Advancing Deep Learning Cancer Classification with application to MRI Liver Tumor Differentiation

Deep Learning (DL) architectures have opened new horizons in medical image analysis attaining unprecedented performance in tasks such as tissue classification and segmentation as well as prediction of several clinical outcomes. In this paper, we propose and evaluate a novel 3D convolutional neural network (CNN) designed for tissue classification in medical imaging and applied for discriminating between primary and metastatic liver tumors from Diffusion Weighted MRI (DWMRI) data. The proposed network consists of four consecutive strided 3D Convolutional layers with 3x3x3 kernel size and ReLU as activation function, followed by a fully connected layer with 2048 neurons and a Softmax layer for binary classification. A dataset composing of 130 DW-MRI scans was used for training and validation of the network. To the best of our knowledge this is the first DL solution for the specific clinical problem and the first 3D CNN for cancer classification operating directly on whole 3D tomographic data without the need of any preprocessing step such as region cropping, annotating or detecting regions of interest. The classification performance results, 83% (3D) vs 69.6% and 65.2% (2D), demonstrated significant tissue classification accuracy improvement comparing to two 2D CNNs of different architectures also designed for the specific clinical problem with the same dataset. These results suggest that the proposed 3D CNN architecture can bring significant benefit in DW-MRI liver discrimination and potentially, in numerous other tissue classification problems based on tomographic data, especially in size-limited, disease specific clinical datasets.