A Bottom-Up Approach for Real-Time Mitral Valve Annulus Modeling on 3D Echo Images

3D+t Transesophageal Echocardiography (TEE) performs 4D scans of mitral valve (MV) morphology at frame rate providing real-time guidance for catheter-based interventions for MV repair and replacement. A key anatomical structure is the MV annulus, and live quantification of the dynamic annulus at acquisition rates 15 fps or higher have proven to be technically challenging. In this paper, we propose a bottom-up approach inspired by clinicians’ manual workflow for MV annulus modeling on 3D+t TEE images in real time. Specifically, we first detect annulus landmarks with clear 3D anatomical features via agents trained using Deep Reinforcement Learning. Leveraging the circular structure of the annulus, cross-annular planes are extracted and additional landmarks are then detected through 2D image-to-image networks on the 2D cutting planes. The complete 3D annulus is finally fitted through all detected landmarks using Splines. We validate the proposed approach on 795 3D+t TEE sequences with 1906 annotated frames, and achieve a speed 20 fps with a median accuracy 2.74 mm curve-to-curve error. Furthermore, device simulation is utilized to augment the training data that results in promising accuracy improvement on challenging echos with visible devices and warrants further investigation.