Deep Reinforcement Learning Based Caching Placement and User Association for Dynamic Cellular Networks

In cache-enabling cellular networks, we investigate the caching placement and content delivery. To cope with the time-varying content popularity and user location in practical scenarios, we formulate a long-term joint dynamic optimization problem of caching placement and user association for minimizing the content delivery delay. We decompose the optimization problem into two sub-problems, the user association sub-problem in short time-scale and the caching placement in long timescale. Specifically, we propose a low complexity belief propagation based user association algorithm in the short time-scale. Then we develop a deep deterministic policy gradient based caching placement algorithm in the long time-scale. Finally, we propose a joint user association and caching placement algorithm to obtain a sub-optimal solution for the proposed problem. We demonstrate the convergence and performance of the proposed algorithm by simulation results. Simulation results show that compared with the benchmark algorithms, the proposed algorithm reduces the long-term content delivery delay in dynamic networks effectively.