Resource Allocation in UAV-Assisted Wireless Networks Using Reinforcement Learning

In this work, we consider the downlink of an unmanned aerial vehicle (UAV) assisted cellular network consisting of multiple cooperative UAVs, whose operations are coordinated by a central ground controller using the fronthaul communications, to serve multiple ground users. A problem of jointly designing UAV's location, transmit beamforming, as well as UAV-user association is formulated in the form of mixed integer nonlinear programming (MINLP) to maximize the sum user achievable rate while considering the constraints of limited fronthaul capacity. Solving the formulated problem is computationally hard owing to the its non-convex nature and the unavailability of channel state information (CSI) due to the undetermined and flexible movement of UAVs. To tackle these effects, we propose a novel algorithm exploiting the deep Q-learning approach to take the hassles of unavailable CSI for determining UAV's location and invoking the difference of convex (DC) based optimization method to efficiently solve for the UAV's transmit beamforming and UAV-user association given the determined UAV's location. The algorithm recursively solves the formulated problem until convergence. Numerical results show that our design outperforms the existing work in terms of algorithmic convergence and network performance and achieve a gain of up to 70% compared to the existing algorithms.