Sample-Path Optimal Stationary Policies in Stable Markov Decision Chains with Average Reward Criterion ?

This work concerns discrete-time Markov decision chains with denumerable state and compact action sets. Besides standard continuity requirements, the main assumption on the model is that it admits a Lyapunov function '. In this context the average reward criterion is analyzed from the sample-path point of view. The main conclusion is that, if the expected average reward associated to ' 2 is nite under any policy, then a stationary policy obtained from the optimality equation in the standard way is sample-path average optimal in a strong sense.