Resource Allocation in Energy Harvesting Multiple Access Scenarios via Combinatorial Learning

The allocation of $K$ orthogonal resources aiming at maximizing the throughput in an energy harvesting (EH) multiple access scenario is considered. In this setting, the optimal resource allocation (RA) depends on the transmitters' EH and channel fading processes. However, in realistic scenarios, only causal knowledge of these processes is available. We first formulate the offline optimization problem and identify two main challenges, namely, how to exploit causal knowledge to maximize the throughput and how to handle the high dimensionality of the problem. To address these challenges, we propose a novel reinforcement learning (RL) algorithm, termed combinatorial RL (cRL). The name stands for its ability to handle the combinatorial nature of the RA solutions. Exploiting the available causal knowledge, we learn the RA policy aiming at maximizing the throughput. Furthermore, we overcome the curse of dimensionality, typical of combinatorial problems, by splitting the learning task, solving $K$ + 1 smaller RL problems and using linear function approximation. Through numerical simulations, we show that cRL outperforms known strategies like random and greedy as well as other RL approaches.