Dynamic handoff decision in heterogeneous wireless systems: Q-learning approach

The satisfaction of a mobile user in a heterogeneous wireless environment relies heavily on the appropriate choice of network. With the presence of various wireless technologies and advances in smart mobile devices, the mobile terminal in next generation wireless communications will likely make intelligent handoff decisions to optimize the user Quality of Experience (QoE). This paper investigates network selection and handoff decision with the goal of maximizing user QoE. An algorithm based on Q-learning is obtained that chooses the best network based not only on the current network state but also the potential future network and device states. As opposed to other dynamic programming-based algorithms, this method does not require the knowledge of the statistics of the wireless environment, but learns an optimum policy by utilizing the mobile device's past experience. It is shown that the QoE results of the proposed Dynamic Handoff Decision (DHD) algorithm come very close to the performance of an optimum oracle algorithm, while on average fewer number of network handoffs are required.