A framework for clinical validation of automatic contour propagation: standardizing geometric and dosimetric evaluation

Abstract Purpose The objective of this work was to outline and demonstrate a standardized framework for evaluating automatically propagated contour quality against expert contours. A two-pronged approach is used to evaluate contour quality: a geometric evaluation to identify geometric and spatial discrepancies between propagated and expert contours, and a comprehensive dosimetric comparison to provide clinical context to the results. Methods and Materials The standardized framework requires a primary image, with reference contours and a radiotherapy treatment plan, and a secondary image. Reference contours are automatically propagated onto the secondary image anatomy and compared to expert contours obtained in an inter-observer study. The standardized framework outlines geometric and dosimetric evaluation methodologies for determining indistinguishability between propagated and expert contours in a cohort analysis. Propagated contours are geometrically compared to expert contours in terms of the Dice Similarity Coefficient (DSC) and the Mean Distance to Agreement (MDA). Statistical analysis is performed on the central tendency and variability of DSC and MDA values over the patient cohort. Dosimetric evaluation involves computing the mean and 95% confidence intervals for the differences in cumulative dose-volume histograms for propagated and expert contours. A case study in accelerated partial breast irradiation was shown to demonstrate the framework. Results The standardized framework was applied to a case study of 24 patient datasets with three radiation oncologists providing the expert contours. Cohort analysis demonstrated that propagated contours were geometrically indistinguishable and dosimetrically distinguishable from expert contours. Conclusions The recommended framework standardizes the comparison of geometric and dosimetric parameters to demonstrate indistinguishability of propagated contours from expert contours. Adoption of this framework is vital for consistent and comprehensive validation of automatic contour propagation for use in large-scale cohort analyses.