Optimized retinal nerve fiber layer segmentation based on optical reflectivity and birefringence for polarization-sensitive optical coherence tomography

Segmentation of the retinal nerve fiber layer (RNFL) from swept source polarization-sensitive optical coherence tomography (SS-PSOCT) images is required to determine RNFL thickness and calculate birefringence. Traditional RNFL segmentation methods based on image processing and boundary detection algorithms utilize only optical reflectivity contrast information, which is strongly affected by speckle noise. We present a novel approach to segment the retinal nerve fiber layer (RNFL) using SS-PSOCT images including both optical reflectivity and phase retardation information. The RNFL anterior boundary is detected based on optical reflectivity change due to refractive index difference between the vitreous and inner limiting membrane. The posterior boundary of the RNFL is a transition zone composed of birefringent axons extending from retinal ganglion cells and may be detected by a change in birefringence. A posterior boundary detection method is presented that segments the RNFL by minimizing the uncertainty of RNFL birefringence determined by a Levenberg-Marquardt nonlinear fitting algorithm. Clinical results from a healthy volunteer show that the proposed segmentation method estimates RNFL birefringence and phase retardation with lower uncertainty and higher continuity than traditional intensity-based approaches.