A Gaussian Process Model Based Generative Framework for Data Augmentation of Multi-modal 3D Image Volumes

Medical imaging protocols routinely employ more than one image modality for diagnosis or treatment. To reduce healthcare costs in such scenarios, research is ongoing on synthetically generating images of one modality from another. Machine learning has shown great potential for such synthesis but performance suffers from scarcity of high quality, co-registered, balanced, and paired multi-modal data required for the training. While methods that do not depend on paired data have been reported, image quality limitations persist including image blurriness. We propose a framework to synthetically generate co-registered and paired 3D volume data using Gaussian process morphable models constructed from a single matching pair of multi-modal 3D image volumes. We demonstrate the application of the framework for matching pairs of CT and MR 3D image volume data with our main contributions being: 1) A generative process for synthesising valid, realistic, and co-registered pairs of CT and MR 3D image volumes, 2) Evaluation of the consistency of the coupling between the generated image volume pairs. Our experiments show that the proposed method is a viable approach to data augmentation that could be used in resource limited environments.