Real-time temporal coherent left ventricle segmentation using convolutional LSTMs

Most work on left ventricle (LV) ultrasound image segmentation using deep learning has focused on single-frame segmentation of end-diastole (ED) and end-systole (ES) frames. Using these neural network models on the entire cardiac cycle often results in segmentation flickering and sudden large segmentation errors. Neural networks that perform some form of temporal reasoning is needed to solve these issues. In this work, we have investigated the use of neural networks with convolutional long short-term memory (ConvLSTM) layers for real-time temporal coherent LV segmentation. A comparison on a dataset of 174 apical 4-, 3- and 2-chamber ultrasound recordings indicated that increasing the number of frames annotated from the cardiac cycle improves temporal segmentation, while using weighted moving average post processing can reduce segmentation flickering, and using ConvLSTM layers reduces large temporal errors considerably. The runtime of the ConvLSTM segmentation network was 13 ms when used in a real-time application for automatic ejection fraction.