ANALYSIS OF MARKOV CHAINS USING SIMULATION GRAPH MODELS

Markov chain analysis is a valuable tool in studying a wide range of problems. Usefully detailed models, however, are usually too cumbersome to be represented as Markov chains. In this paper, we demonstrate the construction of Simulation Graph Models of Markov chains. This approach enables one to evaluate the chain either numerically through simulation or analytically through path analysis. This method can be used in conjunction with simulation to address such problems as rare event estimation, initialization bias, and determination of initial conditions.