Robust Layer Segmentation Against Complex Retinal Abnormalities for en face OCTA Generation

On en face optical coherence tomography angiography (OCTA), the vascular patterns from superficial vascular complex (SVC) and deep vascular complex (DVC) are distinguishable, while outer retina is normally avascular. To visualize en face OCTA images of different vascular patterns, it is inevitable to segment the three regions. However, the automated layer segmentation still faces huge challenge towards manifold advanced tissue lesions affected eyes. In this paper, we first design a region segmentation based augmented 3D U-Net network to fuse spectral domain optical coherence tomography (SD-OCT) structural information and OCTA vascular distribution. Subsequently, an innovative multitask layer-by-layer recoding module breaks up voxel-wise region segmentation probability maps into independent refinement task aiming at further weakening the influence of retinal abnormal regions on layer segmentation. In the end, a simple and effective layer surface encoding module converts the refined region segmentation result of each layer to its continuous surface vector, which advantages are that eliminates the outlier error segmentation in region segmentation tasks and guarantees the uniqueness and strict order constraint of each retinal layer surface in each column. The model validation is carried out on 262 eyes, including 95 normal eyes and 167 multifarious abnormalities affected eyes. The experimental results demonstrate that our method achieves higher segmentation accuracy and stronger ability to fight diseases compared with state-of-the-art segmentation methods.