On Continuity of Transition Probabilities in Belief MDPs with General State and Action Spaces

Natural conditions sufficient for weak continuity of transition probabilities in belief MDPs (Markov decision processes) were established in our paper published in Mathematics of Operations Research in 2016. In particular, the transition probability in the belief MDP is weakly continuous if in the original MDP the transition probability is weakly continuous and the observation probability is continuous in total variation. These results imply sufficient conditions for the existence of optimal policies in POMDPs (partially observable MDPs) and provide computational methods for finding them. In this paper we show that the assumptions from our 2016 paper imply that the transition probability in the belief MDP is continuous in the Wasserstein metric. Since we do not assume compactness of the state space, this is a stronger property than weak continuity.