Adding noise to Markov cohort state-transition models

Abstract Following its introduction over thirty years ago, the Markov cohort model state-transition has been used extensively to model population trajectories over time (Markov trace) in decision modeling and cost-effectiveness studies. We recently showed that a cohort model represents the average of a continuous-time stochastic process on a multidimensional integer lattice governed by a master equation (ME), which represents the time-evolution of the probability function of an integer-valued random vector. Leveraging this theoretical connection, this study introduces an alternative modeling method using a stochastic differential equation (SDE) approach which captures not only the mean behavior but also the variance of the process. The SDE method constitutes the time-evolution of a random vector of population counts. We show the derivation of an SDE model from first principles by postulating the expected change in the distribution of population across states in a small time step. We then describe an algorithm to construct an SDE and solve the SDE via simulation for use in practice. We show the applications of SDE in two case studies. The first example demonstrates that the population trajectories, and their mean and variance, from the SDE and other commonly-used methods match. The second example shows that users can readily apply the SDE method in their existing works without the need for additional inputs beyond those required for constructing a conventional cohort model. In addition, the second example demonstrates that the SDE model is superior to a microsimulation model in terms of computational speed. In summary, an SDE model provides an alternative modeling framework which includes information on variance, can accommodate for time-varying parameters, and is computationally less expensive than microsimulation for a typical modeling problem in cost-effectiveness and decision analyses.