Intelligent fast cell association scheme based on deep Q-learning in ultra-dense cellular networks

To support dramatically increased traffic loads, communication networks become ultra-dense. Traditional cell association (CA) schemes are time-consuming, forcing researchers to seek fast schemes. This paper proposes a deep Q-learning based scheme, whose main idea is to train a deep neural network (DNN) to calculate the Q values of all the state-action pairs and the cell holding the maximum Q value is associated. In the training stage, the intelligent agent continuously generates samples through the trial-and-error method to train the DNN until convergence. In the application stage, state vectors of all the users are inputted to the trained DNN to quickly obtain a satisfied CA result of a scenario with the same BS locations and user distribution. Simulations demonstrate that the proposed scheme provides satisfied CA results in a computational time several orders of magnitudes shorter than traditional schemes. Meanwhile, performance metrics, such as capacity and fairness, can be guaranteed.