Resource Allocation for IRS Assisted SGF NOMA Transmission: A MADRL Approach

Non-orthogonal multiple access (NOMA) assisted semi-grant-free (SGF) transmission has been viewed as one of the promising technologies to meet massive connectivity requirements of the next-generation networks. A novel intelligent reconfigurable surface (IRS) assisted SGF NOMA transmission system is proposed, where the IRS is employed to satisfy the channel gain requirements for grant-based users (GBUs) and grant-free users (GFUs). The dynamic optimization on the sub-carrier assignment and power allocation for roaming GFUs, and the amplitude control and phase shift design for reflecting elements of the IRS, is formulated. Aiming at maximizing the long-term data rate of all GFUs, the optimization problem is first modeled as a multi-agent Markov decision problem. Then, three multi-agent deep reinforcement learning based frameworks are proposed to solve the problem under three different IRS cases, including the ideal IRS, non-ideal IRS with continuous phase shifts, and non-ideal IRS with discrete phase shifts. Specifically, for each GFU agent, a sub-carrier assignment deep Q-network (DQN) and a power allocation deep deterministic policy gradient (DDPG) are integrated to dynamically assign network resources for each GFU. For the only IRS agent, two DDPGs are integrated to dynamically assign phase shift and amplitude for each reflecting element of ideal IRS. The single DDPG for dynamically assigning continuous phase shifts, and parallel DQNs for dynamically assigning discrete phase shifts for non-ideal IRS with fixed amplitude are also proposed. Simulation results demonstrate that: 1) The network sum rates of all GFUs can achieve a significant improvement with the aid of IRS, comparing with the system without IRS. 2) The network sum rates of the NOMA assisted SGF transmissions are superior to that of OMA assisted GF transmissions.