Optimization of Resource Allocation in Multi-Cell OFDM Systems: A Distributed Reinforcement Learning Approach

In this paper, the problem of joint subcarrier and power allocation is studied for multi-cell orthogonal frequency-division multiplexing (OFDM) systems. This joint subcarrier and power resource allocation problem is formulated as an optimization problem whose goal is to maximize the system spectral efficiency. To solve the proposed problem, the original optimization problem is first decomposed into two subproblems: subcarrier allocation and power allocation. By solving these two subproblems, an initial subcarrier and power allocation scheme is accordingly obtained. An multi-agent reinforcement learning (MARL) algorithm is proposed to further increase the spectral efficiency. In particular, using the proposed MARL algorithm, each BS can adapt its allocation scheme according to the wireless environmental states. Numerical results show that the proposed MARL method can achieve up to 53.6% gain in terms of spectral efficiency compared to the conventional scheme. The proposed MARL scheme also converges more rapidly than the conventional single-agent Q-learning approach.