Practical and Dynamic Buffer Sizing using LearnQueue

Wireless networks are undergoing an unprecedented revolution in the last decade. With the explosion of delay-sensitive applications usage on the Internet (i.e., online gaming, VoIP, and safety-critical applications), latency becomes a major issue for the development of wireless technology since it has an enormous impact on user experience. In fact, in a phenomenon known as bufferbloat, large static buffers inside the network devices results in increasing the time that packets spend in the queues and, thus, causing larger delays. Concerns have arisen about designing efficient queue management schemes to mitigate the effects of over-buffering in wireless devices. In this paper, we advocate the exploitation of machine learning techniques for dynamic buffer sizing. We propose LearnQueue, a novel reinforcement learning design that can effectively control the latency in wireless networks. LearnQueue adapts quickly and intelligently to changes in the wireless environment using a sophisticated reward structure. The latency control is performed dynamically by tuning the buffer size. Adopting a trial-and-error approach, the proposed scheme penalizes the actions resulting in longer delays or hurting the throughput. In addition, the scheme parameters are designed for an optimized operation depending on different applications requirements. Using the latest generation of WARP hardware, we investigated LearnQueue performance in various network scenarios. The testbed results prove that LearnQueue can grantee low latency while preserving throughput under various congestion situations. We also discuss the feasibility and possible limitations of large-scale deployment of the proposed scheme in wireless devices.