Energy-aware resource management for uplink non-orthogonal multiple access: Multi-agent deep reinforcement learning

Abstract Non-orthogonal multiple access (NOMA) is one of the promising technologies to meet the huge access demand and the high data rate requirements of the next generation networks. In this paper, we investigate the joint subchannel assignment and power allocation problem in an uplink multi-user NOMA system to maximize the energy efficiency (EE) while ensuring the quality-of-service (QoS) of all users. Different from conventional model-based resource allocation methods, we propose two deep reinforcement learning (DRL) based frameworks to solve this non-convex and dynamic optimization problem, referred to as discrete DRL based resource allocation (DDRA) framework and continuous DRL based resource allocation (CDRA) framework. Specifically, for the DDRA framework, we use a deep Q network (DQN) to output the optimum subchannel assignment policy, and design a distributed and discretized multi-DQN based network to allocate the corresponding transmit power of all users. For the CDRA framework, we design a joint DQN and deep deterministic policy gradient (DDPG) based network to generate the optimal subchannel assignment and power allocation policy. The entire resource allocation policies of these two frameworks are adjusted by updating the weights of their neural networks according to feedback of the system. Numerical results show that the proposed DRL-based resource allocation frameworks can significantly improve the EE of the whole NOMA system compared with other approaches. The proposed DRL based frameworks can provide good performance in various moving speed scenarios through adjusting learning parameters.