Adding noise to Markov cohort models

Following its introduction over thirty years ago, the Markov state-transition cohort model has been used extensively to model population trajectories over time in decision modeling and cost-effectiveness studies. We recently showed that a cohort model represents the average of a continuous-time stochastic process on a multidimensional integer lattice governed by a master equation (ME), which represents the time-evolution of the probability function of a integer-valued random vector. From this theoretical connection, this study introduces an alternative modeling method, stochastic differential equation (SDE), which captures not only the mean behavior but also the variance. We first derive the continuous approximation to the master equation by relaxing integrality constraint of the state space in the form of Fokker Planck equation (FPE), which represents the time-evolution of the probability function of a real-valued random vector. Instead of working with the FPE, the SDE method constitutes time-evolution of the random vector of population counts. We derive the SDE from first principles and describe an algorithm to construct an SDE and solve the SDE via simulation for use in practice. We show the applications of SDE in two case studies. The first example demonstrates that the population trajectories, the mean and the variance, from the SDE and other commonly-used methods match. The second examples shows that users can readily apply the SDE method in their existing works without the need for additional inputs. In addition, in both examples, the SDE is superior to microsimulation in terms of computational speed. In summary, the SDE provides an alternative modeling framework which includes information on variance and is computationally less expensive than microsimulation for a typical modeling problem in cost-effectiveness analyses and in decision analyses.