Pareto-optimal plans as ground truth for validation of a commercial system for knowledge-based DVH-prediction

Abstract Purpose Treatment plans manually generated in clinical routine may suffer from variations and inconsistencies in quality. Using such plans for validating a DVH prediction algorithm might obscure its intrinsic prediction accuracy. In this study we used a recently published large database of Pareto-optimal prostate cancer plans to assess the prediction accuracy of a commercial knowledge-based DVH prediction algorithm, RapidPlan. The database plans were consistently generated with automated planning using an independent optimizer , and can be considered as aground truth of plan quality. Methods Prediction models were generated using training sets with 20, 30, 45, 55 and 114 Pareto-optimal plans. Model-20 and Model-30 were built using 5 groups of randomly selected training patients. For 60 independent Pareto-optimal validation plans, predicted and database DVHs were compared. Results For model-114, differences between predicted and database mean doses of more than ± 10% in rectum, anus and bladder, occurred for 23.3%, 55.0%, and 6.7% of the validation plans, respectively. For rectum V 65Gy and V 75Gy , differences outside the ±10% range were observed in 21.7% and 70.0% of validation plans, respectively. For 61.7% of validation plans, inaccuracies in predicted rectum DVHs resulted in a deviation in predicted NTCP for rectal bleeding outside ±10%. With smaller training sets the DVH prediction performance deteriorated, showing dependence on the selected training patients. Conclusion Even when analysed with Pareto-optimal plans with highly consistent quality, clinically relevant deviations in DVH predictions were observed. Such deviations could potentially result in suboptimal plans for new patients. Further research on DVH prediction models is warranted.