Definition of new WAN paradigms enabled by smart measurements

Nowadays massive amounts of data are being moved over the Internet thanks to data-hungry applications, Big Data, and multimedia content. Combined with a reduction in cost and augmented reliability for high-speed broadband access, the whole Internet infrastructure is facing new challenges especially when information crosses long geographical distances. That is the case for Wide Area Networks (WANs), which are typically traversed in enterprises with multi-site deployments. When a connection is established between end-points that are geographically distant with high latency and high bandwidth, data is flowing over a so-called Long Fat Network. Currently, transport protocols in end-points are not able to exploit the resources of such links, notably the most common TCP implementations still stuffer from design flaws that limit their efficiency. More recent developments still suffer from low fairness in resource sharing and lack of global visibility. We identify SD-WAN as an SDN use-case that can enable new transport protocols adoption, improving traffic behavior over WANs, without the need to modify the end-points. In this thesis, we explore new approaches to network measurements that will enable both end-points and SD-WAN edge routers, to gain visibility over the end-to-end network status. Such additional visibility promotes the development of smarter control mechanisms for network traffic. The preliminary study carried on comprises TCP behavior over WANs and existing methodologies to control its traffic patterns and enforce rate throttling. We also identify a specific use case that poses challenges for WAN scenarios: the Split TCP connections in a Performance Enhancing Proxy (PEP). New control mechanisms to improve resource utilization and fairness are defined in this project. Specifically, we propose a new approach called Receive Window Modulation (RWM) that allows edge-routers to control the sending rate of a TCP connection by modifying the window advertised by the receiver. We prove that such a controller can improve TCP efficiency and fairness by leveraging local information and additional contextual information obtained from network measurements. It also provides a lossless throttling mechanism, allowing for policy enforcement without hindering TCP throughput. We validate RWM in a real experimental scenario, showing improvements of up to 70% in TCP throughput when coupled with loss-based congestion controls. Bufferbloat is also mitigated, reducing the end-to-end TCP latency measured almost three-fold in some scenarios. Another contribution of this project includes a new method to estimate network available bandwidth from TCP passive probings based on the statistical analysis of the Inter-Packet arrival time (SABES). The methodology is based on the packet dispersion model and takes advantage of state-of-the-art machine learning techniques to improve its accuracy, including Deep Neural Networks and Kernel Density Estimation. We validate the model in both simulations and real-world experiments, obtaining a median of the mean absolute error distribution of less than 10% of the network capacity. We study network capacity estimation and bottleneck detection with an innovative active probing approach called HIRE. We propose a new packet dispersion model that takes into account the packet pairs delay, allowing for precise end-to-end capacity estimation. HIRE also introduces the concept of Hidden packets Red-shift Effect, which consists of injecting TTL expiring packets in between probing pairs at a specific rate. This technique allows locating the narrow link position along the path. We validate the model in simulations obtaining an estimation error of less than 3% in most scenarios. All these contributions constitute the building blocks of a Stateful Edge Router Architecture, SERA. Such architecture is presented in the final part of the dissertation, preparing the ground for future developments.