Q-SQUARE: a Q-learning approach to provide a QoE aware UAV flight path in cellular networks

Abstract This paper deals with the adoption of Unmanned Aerial Vehicles (UAVs) as mobile Base Stations providing video streaming services within a cellular macro area. We devise a Q-learning based UAV flight planning algorithm aimed at improving the Quality of Experience (QoE) of video users. Specifically, the proposed algorithm, herein denoted as Q-SQUARE, leverages the well-established Q-learning algorithm by introducing a reward related to a key QoE metric that is the video segment delay. The Q-SQUARE algorithm also accounts for different UAV recharging stations being available in the covered area. The performance analysis, as a function of the number of UAVs and recharging stations, show that Q-SQUARE identifies the UAV flight paths, i.e. specific space-time allocation of the available bandwidth resources, that definitely improve the QoE of the streaming services.