DQ-RM: Deep Reinforcement Learning-based Route Mutation Scheme for Multimedia Services

Increasingly growing various multimedia services (e.g., interactive live video and so on) have brought tremendous pressure on existing static defense techniques. To cope with inherent drawback of static defense techniques, Network Moving Target Defense (NMTD) such as route mutation (RM) was proposed. What's more, applying reinforcement learning (RL) into RM has been proved feasible in our previous work. But two main problems still need to be considered in this combination of RL with RM: 1) It lacks the consideration of multiple flows situation. 2) With the state-action space grow larger, current solution can't handle efficiently. In this paper, we propose a deep Q-learning method for RM (DQ-RM) to solve above two problems. Firstly, benefited from the satisfiability module theory, we formalize RM space considering single flow and multiple flows concurrently. Then we further propose a deep reinforcement learning-based RM scheme based on our previous work, which is suitable for large-scale state-action space. Finally, extensive experimental results highlight the improvement of DQ-RM in defense performance and convergence speed compared to the representative solution.