 |
Deep TCP-RL for congestion control has been explored in
[5], employing the Aurora algorithm against TCP to increase
the throughput by trying to distinguish between packet loss
and congestion, utilizing Neural Networks. This was done by
developing a framework utilizing Open AI Gym Environment.
The two most important parameters which were considered
was history length - how much in the past the agent would
go based on data retention, and discount factor - which
showed that with a higher discount factors, good results are
achieved quickly. Simulation results using Mininet showed
that Aurora was comparable to or outperformed many state
of the art congestion control algorithms such as BBR, PCCVivace,
RemyCC, and Copa. The reward function utilized was
a linear function that rewarded throughput and penalized loss
and latency with coefficients used to force the model to balance
throughput and latency. In the analysis, it was found that TCP
Cubic achieves higher throughput at the expense of higher
latency, however it is plagued with bad performance in many
different scenarios [6]. In [7], the best Initial Window value
and Congestion Control scheme is determined to be used by
RL, based on network characteristics. In [8] both Supervised
Learning and RL is used, with received ACKs, state (e.g. RTT)
as the RL feature input.
In [9], a framework was also presented utilizing Long-Short
Term Memory (LSTM) in a Deep RL (DRL) framework for
MPTCP, in a physical network, which provides benefit to
user traffic by utilizing multiple network connectivity (e.g.
Ethernet, Wi-Fi, and LTE) to load balance a single TCP flow.
The results of this research shows that the use of this type of
framework outperforms traditional MPTCP congestion control
algorithms. While this work does not focus on data centres, it
shows the promise of TCP-RL and further confirms the use of
Q-Learning as the datacenter environment is more predictable
than utilizing multiple network connectivity points, but still
very critical to increasing throughput.
Other works have been done to optimize TCP for data center
networking [10]. In this paper, D-TCP estimates the variable
bandwidth, and based on end-host information, quickly adapts
to changing environments where traditional TCP is not sufficient
for the modern era of network traffic demands.
Work in Omniscient TCP [11] further demonstrates the
inadequacy of TCP to operate in low-latency, high throughput
environments. OTCP in particular achieves a higher level of
performance than standard TCP by configuring TCP congestion
parameters on a route by route basis based on network
characteristics.
Lastly in [12], which utilizes the TCP-RL algorithm, it was
proven that QTCP operated better than TCP NewReno, further
giving momentum to our research.
|
IEEE/ICACT20210131 Slide.07
[Big Slide]