Stabilizing mmWave Backhaul Energy Efficiency for Vehicle-Mounted Access Points by Q-Learning-Based Scheme

Due to the potential for multi-gigabit data rate, millimeter wave (mmWave) bands can play a crucial role in a backhaul link between a small base station (SBS) and a vehicle-mounted access point (AP). Due to the susceptibility to blockage, high path loss, and channel uncertainty at mmWave bands as well as the mobility of vehicle-mounted APs, it is very difficult to stabilize its energy efficiency. We address this challenge by proposing a Q-learning-based scheme, which can stabilize the mmWave moving backhaul energy efficiency in a certain range and avoid frequent network parameters adjustment due to the pursuit of the best. We design the five algorithms for our scheme, where the four algorithms related to Q-learning are executed by each vehicle-mounted AP in an independent mode and the other one is executed by an access controller in a centralized manner. With the help of the access controller, each AP can adjust its network parameters in time to keep the backhaul energy efficiency stable. The simulation results show that the proposed scheme can guarantee almost 100% backhaul link connectivity and basically stable average energy efficiency for all the backhaul links.