Comparison of Statistical Machine Learning Models for Rectal Protocol Compliance in Prostate External Beam Radiation Therapy

Purpose: Limiting the dose to the rectum can be one of the most challenging aspects of creating a dosimetric external beam radiation therapy (EBRT) plan for prostate cancer treatment. Rectal sparing devices such as hydrogel spacers offer the prospect of increased space between the prostate and rectum, causing reduced rectal dose and potentially reduced injury. This study sought to help identify patients at higher risk of developing rectal injury based on estimated rectal dosimetry compliance prior to the EBRT simulation and planning procedure. Three statistical machine learning methods were compared for their ability to predict rectal dose outcomes with varied classification thresholds applied. Methods: Prostate cancer patients treated with conventionally fractionated EBRT to a reference dose of 74–78 Gy were invited to participate in the study. The dose volume histogram data from each dosimetric plan was used to quantify planned rectal volume receiving 50%, 83% 96%, and 102% of the reference dose. Patients were classified into two groups for each of these dose levels: either meeting tolerance by having a rectal volume less than a clinically acceptable threshold for the dose level (Y) or violating the tolerance by having a rectal volume greater than the threshold for the dose level (N). Logistic regression, classification and regression tree, and random forest models were compared for their ability to discriminate between class outcomes. Performance metrics included area under the receiver operator characteristic curve (AUC), sensitivity, specificity, positive predictive value and negative predictive value. Finally, three classification threshold levels were evaluated for their impact on model performance. Results: A total of 176 eligible participants were recruited. Variable importance differed between model methods. Area under the receiver operator characteristic curve performance varied greatly across the different rectal dose levels and between models. Logistic regression performed best at the 83% reference dose level with an AUC value of 0.844, while random forest demonstrated best discrimination at the 96% reference dose level with an AUC value of 0.733. In addition to the standard classification probability threshold of 50%, the clinically representative threshold of 10%, and the best threshold from each AUC plot was applied to compare metrics. This showed that using a 50% threshold and the best threshold from the AUC plots yields similar results. Conversely, applying the more conservative clinical threshold of 10% maximized the sensitivity at V83_RD and V96_RD for all model types. Based on the combination of the metrics, logistic regression would be the recommendation for rectal protocol compliance prediction at the 83% reference dose level, and random forest for the 96% reference dose level, particularly when using the clinical probability threshold of 10%. Conclusions: This study demonstrated the efficacy of statistical machine learning models on rectal protocol compliance prediction for prostate cancer EBRT dosimetric planning. Both logistic regression and random forest modeling approaches demonstrated good discriminative ability for predicting class outcomes in the upper dose levels. Application of a conservative clinical classification threshold maximized sensitivity and further confirmed the value of logistic regression and random forest models over classification and regression tree.