bwNetFlow: A Customizable Multi-Tenant Flow Processing Platform for Transit Providers

In times of increasing bandwidth demands, network operators strive for increased visibility of their network's utilization as well as an indication of the legitimacy of traffic processed across network nodes. Additionally, the detection and mitigation of illegitimate or malicious traffic such as denial of service attacks remains a current and persistently active field of research. Flow-based network monitoring can provide this information live from any network interface. This paper introduces a flow processing platform meant to receive flow information from border interfaces and distribute the acquired information to specialized applications. Transit providers deploying our platform can use this information directly, but also provide all interested customers or network entities with the specific subset concerning them. Between collecting and redistributing the flow information, our platform offers different methods of enrichment using a variety of sources allowing for views superior to plain Netflow records. However, a provided tool can reencode and reexport standard Netflow to ensure compatibility and allow for seamless integration of customer-specific streams into preexisting setups. This platform's components allow the enrichment, division and anonymization of flow data to a number of highly customized streams for any type of application, either on a customer-specific or a network-wide provider level. Applications include the conversion of flow data for time-series databases and the accompanying dashboards, the detection of DDoS attacks or other high-traffic situation on any network level, the identification of faulty network routing policies, or any other use case conceivable on regular flow data, but within an arbitrary network scope.