Stabilizing Transmission Capacity in Millimeter Wave Links by Q-Learning-Based Scheme

Due to uncontrollable factors (e.g., radio channel quality, wireless terminal mobility, and unpredictable obstacle emergence), a millimeter wave (mmWave) link may encounter some problems like unstable transmission capacity and low energy efficiency. In this paper, we propose a new transmission capacity stabilization scheme based on the Q-learning mechanism with the aid of edge computing facilities in an integrated mmWave/sub-6 GHz system. With aid of the proposed scheme, an integrated mmWave/sub-6 GHz user equipment (UE) can adjust its transmission power and angle, even choose a relaying UE to stabilize its transmission capacity. Differing from traditional schemes, the proposed scheme is run in edge computing facilities, where any UE only needs to provide its personalized information (e.g., base station discovery, neighboring UEs, working status (i.e., busy and idle), position coordinates, and residual energy level), and then it will receive intelligent and adaptive guidance from edge computing facilities. This facilitates each UE to maintain its transmission capacity stability by adjusting its radio parameters. The simulation results show that any UE with aid of the proposed scheme can achieve more stable transmission capacity and higher energy efficiency.