Deep learning of spatiotemporal filtering for fast super-resolution ultrasound imaging.

Super-resolution ultrasound imaging (SR-US) is a new technique that breaks the diffraction limit and allows visualization of microvascular structures down to tens of microns. Image processing methods for the spatiotemporal filtering needed in SR-US, such as singular value decomposition (SVD), are computationally burdensome and performed off-line. Deep learning has been applied to many biomedical imaging problems and trained neural networks have been shown to process an image in milliseconds. The goal of this study was to evaluate the effectiveness of deep learning to realize a spatiotemporal filter in the context of SR-US processing. A 3D convolutional neural network (3DCNN) was trained on in vitro and in vivo datasets using SVD as ground truth in tissue clutter reduction. In vitro data was obtained from a tissue-mimicking flow phantom and in vivo data was collected from murine tumors of breast cancer. Three training techniques were studied: training with in vitro datasets, training with in vivo datasets, and transfer learning with initial training on in vitro datasets followed by fine-tuning with in vivo datasets. The neural network trained with in vitro datasets followed by fine-tuning with in vivo datasets had the highest accuracy at 88.0%. The SR-US images produced with deep learning allowed visualization of vessels as small as 25 mum in diameter, which is below the diffraction limit (wavelength of 110 lm at 14 MHz). The performance of the 3DCNN was encouraging for real-time SR-US imaging with an average processing frame rate for in vivo data of 51 Hz with GPU acceleration.