Fast and Accurate Traffic Measurement With Hierarchical Filtering

Sketches have been widely used to record traffic statistics using sub-linear space data structure. Most sketches focus on the traffic estimation of elephant flows (i.e., heavy hitters) due to their importance to many network optimization tasks, e.g., traffic engineering and load balancing. In fact, the information of aggregate mice flows (e.g., all the mice flows with the same source IP) is also crucial to many security-associated tasks, e.g., DDoS detection and network scan detection. However, the previous solutions, e.g., measuring each individual flow or using multiple sketches for independent measurement tasks, will result in worse estimation error or higher computational overhead. To conquer the above disadvantages, we propose an accurate traffic measurement framework with multiple filters, called Sketchtree, to efficiently measure both elephant flows and aggregate mice flows. These filters in Sketchtree are organized in a hierarchical manner, and help to alleviate the hash collision and improve the measurement accuracy, as the number of flows through hierarchical filters in turn will be decreased gradually. We also design some mechanisms to improve the resource utilization efficiency. To validate our proposal, we have implemented Sketchtree and conducted experimental evaluation using real campus traffic traces. The experimental results show that Sketchtree can increase the processing speed by 100 percent, and reduce the measurement error by over 30 percent compared with state-of-the-art sketches.