PPO-RM: Proximal Policy Optimization Based Route Mutation for Multimedia Services

The growing multimedia services have brought unprecedented challenges to the traditional static network architecture. Moving Target Defense (MTD) has been proposed to solve the inherent disadvantages of existing defense techniques. As an important area of MTD research, Route Mutation (RM) can dynamically change the forwarding routes in the network. In our previous work, we applied Reinforcement Learning (RL) to RM. However, there are still two problems that need to be addressed. 1) We consider too few constraints to reflect the actual network situation. 2) Due to the slow rate of convergence, it becomes difficult for efficient deployment. In this paper, we propose a Proximal Policy Optimization Based Route Mutation (PPO-RM) scheme to solve these problems. Firstly, we utilize the Satisfiability Module Theory (SMT) to formalize the space of all possible mutated routes. Then, we design an RM algorithm based on Proximal Policy Optimization (PPO) and implement it in the SDN controller. Finally, we simulate on Mininet to validate our method. The experiment results show that PPO-RM achieves the improvement in terms of convergence rate, defense performance, and network performance.