BayCANN: Streamlining Bayesian Calibration With Artificial Neural Network Metamodeling

Bayesian calibration is generally superior to standard direct-search algorithms because it can reveal the full joint posterior distribution of the calibrated parameters. However, there are many barriers to using Bayesian calibration in health decision sciences involving the developmental efforts needed to program complex models in probabilistic programming languages and the associated computational burdens of applying Bayesian calibration. In this paper, we propose to use artificial neural networks (ANN) as one solution to these challenges. Bayesian Calibration using Artificial Neural Networks (BayCANN) involves (1) training an ANN metamodel on a sample of model inputs and outputs, and (2) then calibrating the trained ANN metamodel instead of the full model in a probabilistic programming language to obtain the posterior joint distribution of the calibrated parameters. We illustrate BayCANN using a colorectal cancer natural history model. We conduct a confirmatory simulation analysis by first obtaining parameter estimates from the literature and then using them to generate adenoma prevalence and cancer incidence targets. We compare the performance of BayCANN in recovering these “true” parameter values against performing a Bayesian calibration directly on the simulation model using an incremental mixture importance sampling (IMIS) algorithm. We were able to apply BayCANN using only a dataset of the model inputs and outputs and minor modification of BayCANN’s code. In this example, BayCANN was slightly more accurate in recovering the true posterior parameter estimates compared to IMIS. In addition, because the CRC model was very efficient, the ANN did not offer large computational gain. Obtaining the dataset of samples, and running BayCANN took 15 minutes compared to the IMIS which took 80 minutes. In other examples that may involve computationally more expensive simulations (e.g., microsimulations), we expect BayCANN to offer higher relative speed gains. BayCANN only uses a dataset of model inputs and outputs to obtain the calibrated joint parameter distributions. Thus, it can be adapted to models of various levels of complexity with minor or no change to its structure. In addition, BayCANN’s efficiency can be especially useful in computationally expensive models. To facilitate BayCANN’s wider adoption, we provide BayCANN’s open-source implementation in R and stan.