Microservice Selection in Edge-Cloud Collaborative Environment: A Deep Reinforcement Learning Approach

The hybrid edge/cloud environment requires complex selection strategies to ensure the user’s quality of service. However, current service selection strategies ignore the dynamic and heterogeneous characteristics in an edge-cloud collaborative environment. In this paper, an edge-cloud collaborative network is considered. A deep deterministic policy gradient algorithm for microservice selection called MS_DDPG has been proposed considering user’s mobility, edge server’s and application’s heterogeneity, etc. Microservice selection scheduling (MSSC) problem is defined as an optimization problem of minimizing user’s service access delay. This problem is regarded as a Markov decision-making process, and the microservice selection strategy experience pool is introduced to deal with the complexity of the edge-cloud collaborative environment and improve learning efficiency. MS_DDPG has been compared with two baseline algorithms, MC_ONLINE and RANDOM, with real datasets and some synthetic datasets. The experimental results demonstrate that MS_DDPG is superior to the other two algorithms and has good robustness.