Tensor-Based Reinforcement Learning for Network Routing

In the recent years, we have witnessed an explosion of networking applications due to the reasons such as the rapid development of cloud infrastructure, edge computing, and the Internet of Things. Furthermore, those applications become complex, the problem related to the large size of the state space and limited metric collection has emerged. This leads to an urging demand for adaptive management method in network routing. However, the complexity of traditional routing algorithms can be prohibited for practical systems. To overcome this challenge, we propose a novel tensor-based reinforcement learning method to route and schedule the packet flows, which is adaptive and model-free. Moreover, we improve the learning quality and efficiency by combining the Tucker decomposition technique within the learning process so that the machine learning direction can be obtained with low complexity. Finally, simulation results show that our proposed algorithm can achieve better performance under the same training episode and more stable results with less convergence time than conventional routing method, K-shortest path, traditional reinforcement learning approaches (i.e. Q-learning and SARSA) and comparable results to DQL.