Protecting VNF services with smart online behavior anomaly detection method

Abstract Network Function Virtualization (NFV) is an emerging technology that allows network operators to deploy their Virtualized Network Functions (VNFs) on low-cost commodity servers in the cloud data center. The VNFs, such as virtual routers, firewalls etc., that typically control and transmit critical network packages, require strong security guarantees. However, detecting malicious or malfunctioning VNFs are challenging, as the behaviors of VNFs are dynamic and complex due to the changing network traffics in the cloud. In this paper, we propose a smart and efficient Hidden Markov Model based anomaly detection system (named vGuard) to protect online VNF services in the cloud. A general multivariate HMM model is proposed to profile the normal VNF behavior patterns. Using the VNF behavior model trained with normal observation sequences, vGuard can effectively detect abnormal behaviors online. vGuard is a general framework that can train different types of VNF behavior models. We implement the vGuard prototype in the OpenStack platform. Two types of VNF models, virtual router and virtual firewall, are trained using real normal network traffics in our experiment evaluation. A collection of abnormal attack cases are tested on the VNFs that showed the effectiveness of vGuard in detecting VNF behavior anomalies.