{"id":285,"date":"2011-04-15T22:09:17","date_gmt":"2011-04-15T22:09:17","guid":{"rendered":"http:\/\/www.syslog.cl.cam.ac.uk\/?p=285"},"modified":"2011-04-15T22:09:17","modified_gmt":"2011-04-15T22:09:17","slug":"ietf80-congestion-control","status":"publish","type":"post","link":"https:\/\/www.syslog.cl.cam.ac.uk\/2011\/04\/15\/ietf80-congestion-control\/","title":{"rendered":"IETF 80 highlights: congestion control"},"content":{"rendered":"
There were several interesting talks on various aspects of congestion control at IETF 80, spread around various working groups and research groups; the majority of work that I would classify as actual\u00c2\u00a0research<\/em> being done in the IETF and IRTF at the moment seems to concern congestion control in some way or other. \u00c2\u00a0I've already written about Multipath TCP<\/a> and Bufferbloat<\/a>; here's a potpourri of other TCP problems and proposed solutions. \u00c2\u00a0Most of these came out of the meeting of the\u00c2\u00a0Internet Congestion Control Research Group (ICCRG)<\/a> - strictly part of the IRTF<\/a> rather than the IETF - but the presentation on SPDY came from the IETF Transport Area open meeting.<\/p>\n This work being undertaken by Mirja K\u00c3\u00bchlewind<\/a> and Bob Briscoe<\/a> at BT is a really neat way for TCP to react to changes in available capacity quickly and accurately. \u00c2\u00a0As things stand, if a significant amount of new capacity appears after a connection has left the slow-start (exponential) phase, classic TCP can take a long time to make use of this capacity via additive increase of the congestion window. \u00c2\u00a0The state of the art prior to this work reacts more quickly but will overshoot and cause unnecessary congestion.<\/p>\n Their solution is to \"chirp\"\u00c2\u00a0groups of data packets. \u00c2\u00a0This is an adaptation of an old radio concept<\/a>; in the analogue domain, a chirp is a signal with increasing or decreasing frequency over time. \u00c2\u00a0Here, chirping means to transmit a group of packets with steadily-decreasing inter-packet intervals, or in other words steadily-increasing data rates. \u00c2\u00a0At some point, the sender will pass the capacity of the bottleneck; the part of the chirp sent too quickly for the network to cope with will have been spread out when it reaches the receiver, i.e. the inter-packet intervals will flatten out at a minimum value from which the available capacity can be easily computed.<\/p>\n This application of chirping to networking was originally devised by Ribeiro et al. -\u00c2\u00a0pathChirp [postscript]<\/a> -\u00c2\u00a0as a means for testing a link; K\u00c3\u00bchlewind and Briscoe have implemented this as a congestion control mechanism for TCP, sending every<\/em> user\u00c2\u00a0data packet as part of a chirp (on the order of 32 packets or half a second long) in order to continuously\u00c2\u00a0reevaluate the available bandwidth. \u00c2\u00a0TCP no longer has to rely on loss to detect the optimal window, and nor does it have to fill the buffer next to the bottleneck - no more\u00c2\u00a0buffer-induced latency<\/a>! \u00c2\u00a0A member of the audience commented that this could work\u00c2\u00a0particularly\u00c2\u00a0well on wireless links.<\/p>\n There are a few open research questions, though: for instance it is not clear what will happen when everybody is chirping; it may be that chirps interact with each other at the bottleneck.<\/p>\n Gorry Fairhurst<\/a> of the University of Aberdeen made the case for revisiting an assumption inherent in TCP: that flows have traditionally been considered to be either \"bulk\" (transmitting data as fast as possible) or \"thin\" (sitting idle for most of the time), not both. \u00c2\u00a0More recently we have seen flows which are both or neither, such as audio\/video streams where the transmission rate is governed by the application or persistent HTTP connections which sit idle but switch to bulk operation occasionally.<\/p>\n Standard TCP does very badly with such connections for two reasons:<\/p>\n The proposed fix is to preserve the existing congestion window unmodified (for up to six minutes) whenever the application transmits at less than two-thirds of this window - i.e. the congestion window found during bursts of transmission is tracked. \u00c2\u00a0(The 6-minute timeout is an arbitrary compromise.) \u00c2\u00a0A connection which is idle (or transmitting slowly) for less than six minutes can resume transmission immediately at any time using its previously-determined congestion window.<\/p>\n As noted by Mark Handley, however, this behaviour does carry a risk: it is quite possible for the network conditions - in particular, competing traffic - to change significantly during a sub-six-minute idle period, which could cause the connection to transmit at a wildly-inappropriate rate when it returns from idle leading to massive congestion.<\/p>\n \u00ef\u00bb\u00bf\u00ef\u00bb\u00bfMurari Sridharan of Microsoft is a vociferous proponent for getting the IETF interested in the datacentre, which is approaching and surpassing the limits of several protocols (for example ARP, which is why I was there, but that's another story). \u00c2\u00a0From his perspective the prevailing opinion in the IETF is that the datacentre is \"not the internet\" and tends to be the domain of proprietary equipment running proprietary protocols. \u00c2\u00a0However some of the problems experienced in the datacentre relate specifically to TCP\/IP and may have to be solved by modifying the OS or hypervisor; if that happens, then these modifications will doubtless interact with the internet at some point, either deliberately or accidentally. \u00c2\u00a0Furthermore, as echoed by BT and Google, the line between datacentre and access network is blurring as access technologies increase in bandwidth and datacentres become more distributed.<\/p>\n The specifics of the datacentre from Sridharan's perspective (which are of course biased towards the situation in Microsoft datacentres, but probably apply more generally) are:<\/p>\n On top of this demanding scenario, datacentre operators want high burst tolerance and<\/em> low latency and <\/em>high throughput (it's hard to get all three) - and they also want performance isolation between flows. \u00c2\u00a0TCP is the state of the art in performance isolation, but it works on the wrong granularity for (at least) Microsoft's datacentres. \u00c2\u00a0Simply turning on ECN ubiquitously in the datacentre does improve the situation significantly, but they would like to go further, by specifically addressing the incast and performance isolation problems.<\/p>\n Microsoft has attempted to improve congestion control in two parallel ways: firstly, by using congestion-controlled tunnels between VMs (Seawall [pdf]<\/a>) and secondly by tweaking TCP itself (DCTCP [pdf]<\/a>). \u00c2\u00a0Seawall's encapsulating behaviour is known to break hardware offload, amongst other things; Sridharan went into more detail on DCTCP.<\/p>\n DCTCP aims to make TCP less bursty, and can be thought of as a more-capable ECN: rather than just reacting to the presence of congestion, DCTCP reacts in proportion to its extent<\/em>,\u00c2\u00a0with switches marking packets probabilistically based on instantaneous queue length (actually an estimate of the fraction of time the queue length exceeds a threshold, but this has been shown to be equivalent by Kelly et al). \u00c2\u00a0This higher-resolution data is needed to deal with the incast problem where short-timescale feedback is important. \u00c2\u00a0Like ECN, DCTCP uses a single marker bit as feedback, but spreads more-detailed feedback over multiple packets: if one packet in 20 is marked, that means the congestion window should be cut by 5%.<\/p>\n Interestingly, according to Sridharan, DCTCP can be implemented using existing silicon despite the new congestion marking requirements on switches.<\/p>\n SPDY<\/a> is a replacement for HTTP initiated by Google (and presented by Mike Belshe<\/a>); the motivation is that HTTP is somewhat network-unfriendly in the way it uses TCP, and that it would be better to fix the application layer's abuse of the transport layer before taking the plunge and deciding to adapt or replace TCP.<\/p>\n As background, Belshe described the \"average webpage\", which consists of:<\/p>\n HTTP fetches these 44 resources in an inefficient manner. \u00c2\u00a0SPDY aims to use fewer connections, fewer bytes and fewer packets to fetch the same resources:<\/p>\n SPDY has been implemented in Chrome and is enabled for SSL traffic where the server supports it; since Google's servers do, they have a fair amount of real data on the performance of this new protocol. \u00c2\u00a0In summary (it is claimed) SPDY is a significant improvement over HTTP.<\/p>\n However, there is an open problem: SPDY's use of a single connection hurts it in a few ways, despite this being \"more correct\" behaviour than that of HTTP. \u00c2\u00a0First and foremost, RFC-compliant hosts using an initial congestion window of 1 packet will take a while to reach an acceptable throughput; Google \"solves\" this by setting the initial congestion window on their servers to 10 packets in direct violation of the spec, for which they received a certain amount of flak in the IETF! \u00c2\u00a0Furthermore, Belshe also pointed out that TCP unfairly penalises single-connection protocols: a single packet loss halves SPDY's throughput, whereas in the presence of multiple HTTP connections only one of those will be hurt and the throughput will be reduced by a lesser amount.<\/p>\n SPDY is very much a work in progress and\u00c2\u00a0the developers are looking into ways to avoid this \"single-connection tax\". \u00c2\u00a0Their roadmap also includes a few further enhancements; most interestingly they are experimenting with including request data in the SYN packet to avoid a round trip.<\/p>\n There are (as was pointed out by various audience members) preexisting alternatives to SPDY. \u00c2\u00a0However, SCTP is less deployable on today's internet; Google considers that this is more likely to work if deployed at the application layer on top of TCP. \u00c2\u00a0HTTP pipelining should solve some of the problems, but it interacts badly with some proxies so is generally turned off - and it does not implement prioritisation. \u00c2\u00a0It turns out many hacks are necessary to reliably use pipelining; Firefox is close to having this working by running tests in the background and switching pipelining on as and when they pass.<\/p>\n Finally, a memorable quotation courtesy of Belshe: \"the gentlemen's agreement in congestion control is over\". \u00c2\u00a0This was meant to defend SPDY's and Google's flagrant violation of the TCP specs and replacement of HTTP with a nonstandard protocol, but nevertheless it is food for thought.<\/p>\n","protected":false},"excerpt":{"rendered":" There were several interesting talks on various aspects of congestion control at IETF 80, spread around various working groups and research groups; the majority of work that I would classify as actual\u00c2\u00a0research being done in the IETF and IRTF at the moment seems to concern congestion control in some way or other. \u00c2\u00a0I’ve already written […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[6,23],"tags":[24],"_links":{"self":[{"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/posts\/285"}],"collection":[{"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/comments?post=285"}],"version-history":[{"count":12,"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/posts\/285\/revisions"}],"predecessor-version":[{"id":307,"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/posts\/285\/revisions\/307"}],"wp:attachment":[{"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/media?parent=285"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/categories?post=285"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/tags?post=285"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Chirping for Congestion Control<\/h1>\n
Updating TCP to support Variable-Rate Traffic<\/h1>\n
\n
Datacentre problems and DCTCP<\/h1>\n
\n
SPDY<\/h1>\n
\n
\n