External validation of a prognostic model for mortality among patients with non–small-cell lung cancer using the veterans precision oncology data commons

Abstract Background: There is wide interest in developing prognostic models in non–small-cell lung cancer (NSCLC) due to the heterogeneity of the disease. Models developed at other healthcare institutions may not be directly applicable for patients treated at the Department of Veterans Affairs (VA). External validation of a candidate prognostic model among VA patients would be crucial before it can be implemented to aid clinical decision-making. Methods: A prognostic model for mortality developed in the Military Health System (MHS) was applied to data from the VA Precision Oncology Data Repository (VA-PODR), which is available to researchers inside and outside the VA at the Veterans Precision Oncology Data Commons (VPODC). Measures of discrimination and calibration were calculated for the MHS model. The MHS model was also refitted in VA-PODR data using the same risk factors to compare the effect of specific factors and predictive performance when the model is developed using VA data. Results: Time-dependent AUC of the MHS prognostic model was 0.788, 0.806, 0.780, and 0.779 for predicting survival at 1, 2, 3, and 5 years following diagnosis, respectively. Significant discrepancies were found between predicted and observed rates of survival, particularly for later years. When the model is refit in VA-PODR data, it achieved cross-validated AUCs of 0.739, 0.773, 0.769, and 0.807 at the same time points, and discrepancies between predicted and observed survival were reduced. Conclusions: Validation of the MHS prognostic model in VA-PODR demonstrates that its discrimination remains strong when applied to VA patients. Nevertheless, further calibration to VA data may be needed to improve its risk estimation performance. This study highlights the utility of VA-PODR and the VPODC as a national resource for developing analytic tools that are welladapted to the Veteran population.