Experience-Driven Congestion Control: When Multi-Path TCP Meets Deep Reinforcement Learning
157
Citation
48
Reference
10
Related Paper
Citation Trend
Abstract:
In this paper, we aim to study networking problems from a whole new perspective by leveraging emerging deep learning, to develop an experience-driven approach, which enables a network or a protocol to learn the best way to control itself from its own experience (e.g., runtime statistics data), just as a human learns a skill. We present design, implementation and evaluation of a deep reinforcement learning (DRL)-based control framework, DRL-CC (DRL for Congestion Control), which realizes our experience-driven design philosophy on multi-path TCP (MPTCP) congestion control. DRL-CC utilizes a single (instead of multiple independent) agent to dynamically and jointly perform congestion control for all active MPTCP flows on an end host with the objective of maximizing the overall utility. The novelty of our design is to utilize a flexible recurrent neural network, LSTM, under a DRL framework for learning a representation for all active flows and dealing with their dynamics. Moreover, we, for the first time, integrate the above LSTM-based representation network into an actor-critic framework for continuous (congestion) control, which leverages the emerging deterministic policy gradient to train critic, actor, and LSTM networks in an end-to-end manner. We implemented DRL-CC based on the MPTCP implementation in the Linux kernel. The experimental results show that 1) DRL-CC consistently and significantly outperforms a few well-known MPTCP congestion control algorithms in terms of goodput without sacrificing fairness, 2) it is flexible and robust to highly-dynamic network environments with time-varying flows, and 3) it is friendly to regular TCP.Keywords:
Goodput
In this paper, the expressions for the forced termination probability, blocking probability and aggregate goodput of a single channel SU network with fixed packet size are derived. For a given SU data rate and fixed header length, it has been shown that there exists an optimal payload length for SUs packets which maximizes the aggregate goodput. Using the derived results, the impact on the aggregate goodput by three channel allocation methods i.e., intuitive, optimal, and proposed heuristic is investigated in a multiple channel CR network. Numerical studies show that the goodput of the proposed heuristic method is about 90-95% of that of the optimal method, and about 20-25% better than that of the intuitive method. Simulation results are also included.
Goodput
Header
Payload (computing)
Cite
Citations (3)
The current stream control transmission protocol (SCTP) does not work well over lossy links. To make SCTP congestion control algorithms robust in lossy networks, the paper first introduces a fine-tuned explicit congestion notification (ECN) mechanism for SCTP in such environment, and then discusses the ECN-D SCTP that can differentiate noncongestion losses from congestion losses. We identify the optimal value of the congestion window for an SCTP source in response to ECN messages in order to maximize the throughput and maintain relatively small end-to-end delay. A simple and practical method to achieve the optimal value is developed by carefully choosing the threshold of queues that support ECN. Because the total goodput performance of SCTP associations is not sensitive to window reduction policies when the network load is heavy, and because fine-tuning SCTP or transmission control protocols congestion window in response to congestion indications using complicated methods may not be worth the increase in complexity of the protocol, the simplified method becomes attractive in achieving the optimal congestion window.
Goodput
Explicit Congestion Notification
Lossy compression
Sliding window protocol
Traverse
Cite
Citations (9)
In a wireless mesh network (WMN) connected to the outer infrastructure such as the Internet, one or more gateways to connect to the outer network as well as surrounding nodes may frequently be congested by concentration of traffic from mobile stations. To resolve the congestion, we propose a new congestion control scheme for layer 3 WMNs. In the proposed scheme, nodes composing WMN backbone (MNs: mesh nodes) monitor their transmission queue periodically and detect congestion. When an MN detects congestion, it identifies MNs which inject a dominant amount of traffic causing congestion, and sends a congestion control message to these MNs. When the MNs receive the congestion control message, they control the bandwidth of the traffic toward the congested MN. We have performed simulation of the proposed scheme using a simple network model and evaluated the effect of congestion control. We have shown that the proposed method is effective to reduce average queue length of the bottle-neck MN and to improve the total throughput significantly.
Explicit Congestion Notification
Flow Control
Shared mesh
Packet loss
Cite
Citations (3)
We study the cost of improving the goodput, or the useful data rate, to user in a wireless network. We measure the cost in terms of number of base stations, which is highly correlated to the energy cost as well as capital and operational costs of a network provider.We show that increasing the available bandwidth, or throughput, may not necessarily lead to increase in goodput, particularly in lossy wireless networks in which TCP does not perform well. As a result, much of the resources dedicated to the user may not translate to high goodput, resulting in an inefficient use of the network resources. We show that using protocols such as TCP/NC, which are more resilient to erasures and failures in the network, may lead to a goodput commensurate the throughput dedicated to each user. By increasing goodput, users' transactions are completed faster; thus, the resources dedicated to these users can be released to serve other requests or transactions. Consequently, we show that translating efficiently throughput to goodput may bring forth better connection to users while reducing the cost for the network providers.
Goodput
Linear network coding
Cite
Citations (2)
We study the cost of improving the goodput, or the useful data rate, to user in a wireless network. We measure the cost in terms of number of base stations, which is highly correlated to the energy cost as well as capital and operational costs of a network provider. We show that increasing the available bandwidth, or throughput, may not necessarily lead to increase in goodput, particularly in lossy wireless networks in which TCP does not perform well. As a result, much of the resources dedicated to the user may not translate to high goodput, resulting in an inefficient use of the network resources. We show that using protocols such as TCP/NC, which are more resilient to erasures and failures in the network, may lead to a goodput commensurate the throughput dedicated to each user. By increasing goodput, users’ transactions are completed faster; thus, the resources dedicated to these users can be released to serve other requests or transactions. Consequently, we show that translating efficiently throughput to goodput may bring forth better connection to users while reducing the cost for the network providers.
Goodput
Cite
Citations (0)
In WiMAX networks, the base station (BS) is a likely bottleneck for downlink (DL) TCP connections due to difference in available bandwidth between the fixed network and the wireless link. This may result in buffer overflows or excessive delays at the BS, as these buffers are connection-specific. In order to avoid buffer overflows, different active queue management (AQM) methods may be applied at the BS. This paper presents an analysis of several AQM mechanisms and proves that they are indeed very useful: AQM reduces considerably DL delays at the WiMAX BS without sacrificing TCP goodput.
Goodput
WiMAX
Cite
Citations (24)
Wireless mesh networks (WMNs) cannot utilize the full capability of these radio technologies without an effective channel assignment algorithm. Existing metrics for characterising the efficacy of channel assignment algorithms poorly reflect the goodput distribution in WMNs. In this paper, we develop a fitness evaluation algorithm that feeds back several indicators of efficacy to a channel assignment algorithm and evaluate the resulting goodput distribution. To the best of our knowledge, our algorithm is the first to evaluate the fitness of channel assignment algorithms with goodput distribution, average goodput, and fairness index. Simulation results show that our evaluation algorithm provides accurate prediction of goodput distribution, average goodput, and fairness index with errors below 8%.
Goodput
Cite
Citations (3)
Goodput
Unicast
Cite
Citations (3)
A common practice in the Internet of Things (IoT) is to integrate the Constrained Application Protocol (CoAP), a lightweight application layer protocol based on UDP (User Datagram Protocol), in its architecture. While UDP does not have any provision to control congestion at the transport layer, CoAP and its other variants, such as CoCoA and CoCoA+, get around this issue by employing their own congestion control mechanism, a send-rate-based congestion control technique, at the application layer. These techniques estimate the send-rate through Round Trip Time (RTT), however, fail to map acknowledgments and re-transmitted messages, leading to inaccurate RTT in case of bursty traffic and thus performs poorly. To tackle this problem, the article proposes novel a technique to regulate the send-rate considering the current and previous congestion ratio, congestion factor, and throughput. The proposed method aims to regulate the send-rate according to congestion level in the network. The performance measurements of the proposed approach show significant increments in goodput and fairness as compared to the techniques used in CoCoA+ and CoAP in burst traffic scenarios.
Goodput
Explicit Congestion Notification
Transport layer
Application layer
Cite
Citations (0)