Sufficiency of Markov Policies for Continuous-Time Jump Markov Decision Processes

One of the basic facts known for discrete-time Markov decision processes is that, if the probability distribution of an initial state is fixed, then for every policy it is easy to construct a (rand...