Frame Rate Up-Conversion in Echocardiography Using a Conditioned Variational Autoencoder and Generative Adversarial Model

Accurate detection of heart-related diseases in echocardiography (echo) often requires determining the performance of cardiac valves or contractile events such as strain at a high temporal resolution. In high-end cart-based imaging systems, this is achieved by increasing the frame rate using specialized beamforming and imaging hardware, or by limiting the imaging field of view (FOV). In point-of-care imaging, such a high frame rate imaging technology is currently unavailable. In this paper, we propose a new frame rate up-conversion technique, as a post-processing step during or after the echo acquisition. The proposed technique takes advantage of both variational autoencoders (VAE) and generative adversarial networks (GAN), and produces realistic frames at a high frame rate that can be used to augment conventional imaging. The proposed technique is robust to variations in heart rate since its latent space not only uses immediate previous frames, but it also takes into account the appearance of end-diastolic and end-systolic frames in its estimation. Our results show that the proposed technique can increase the frame rate by at least 5 times without any requirement for limiting the imaging FOV.