Experience-Driven Power Allocation Using Multi-Agent Deep Reinforcement Learning for Millimeter-Wave High-Speed Railway Systems

Railway is stepping into the field of smart railway. Unfortunately, the challenge of obtaining accurate instantaneous channel state information in high-speed railway (HSR) scenario makes it difficult to apply conventional power allocation schemes. In this paper, to respond to the challenge, we propose an innovative experience-driven power allocation algorithm in the millimeter-wave (mmWave) HSR systems with hybrid beamforming, which is capable of learning power decisions from the past experience instead of the accurate mathematical model, just like one person learns one new skill, such as driving. To be specific, with the purpose of maximizing the achievable sum rate, we first characterize the power allocation problem of the mmWave HSR systems as a multi-agent deep reinforcement learning problem and then solve it by using emerging multi-agent deep deterministic policy gradient (MADDPG) approach, which enables the agent, i.e., the mobile relay onboard the train to learn the power decisions from the past experience in a distributed manner. The simulation results indicate that the spectral efficiency of proposed MADDPG algorithm significantly outperforms existing state-of-the-art schemes.