A note on solutions to controlled martingale problems and their conditioning

In this note, we rigorously justify a conditioning argument which is often (explicitly or implicitly) used to prove the dynamic programming principle in the stochastic control literature. To this end, we set up controlled martingale problems in an unusual way.