{"id":1287,"date":"2013-08-14T03:47:37","date_gmt":"2013-08-14T03:47:37","guid":{"rendered":"http:\/\/www.syslog.cl.cam.ac.uk\/?p=1287"},"modified":"2013-08-14T07:14:17","modified_gmt":"2013-08-14T07:14:17","slug":"liveblog-from-sigcomm13-day-2","status":"publish","type":"post","link":"https:\/\/www.syslog.cl.cam.ac.uk\/2013\/08\/14\/liveblog-from-sigcomm13-day-2\/","title":{"rendered":"Liveblog from SIGCOMM13 \u00e2\u20ac\u201c Day 2"},"content":{"rendered":"<p>Hello again from SIGCOMM 2013, Hong Kong, day 2. I'll be live blogging as much of the event as I can, weather permitting...<\/p>\n<p><!--more--><\/p>\n<h1>Morning Sessions Canceled<\/h1>\n<p>The Hong Kong weather observatory has issued a \"Typhoon Warning 8\" due to Tropical Cyclone Utor, which basically means that all government infrastructure has been suspended. This includes the University at which SIGCOMM13 is being held. We are awaiting further updates. Current weather forecasts suggest that this condition will remain until around 2pm today. \u00c2\u00a0More details expected at midday.<\/p>\n<p>&nbsp;<\/p>\n<h1>Datacenter Networking 1<\/h1>\n<h2>Ananta: Cloud Scale Load Balancing<\/h2>\n<p>http:\/\/conferences.sigcomm.org\/sigcomm\/2013\/papers\/sigcomm\/p207.pdf<\/p>\n<ul>\n<li><span style=\"line-height: 13px;\">Desire a load\u00c2\u00a0balancer\u00c2\u00a0that does 20TBbps and has N+1\u00c2\u00a0redundancy\u00c2\u00a0with quick failover.\u00c2\u00a0<\/span><\/li>\n<li>Distributed loadballencer with PAXOS for keeping NAT state synced.<\/li>\n<li>To manage latency, group pools of 8 port and 160 VMs.<\/li>\n<\/ul>\n<p><strong>[mpg39<\/strong><b>: sorry, this is a bit of a poor summary, I was a bit late and the speaker was very\u00c2\u00a0quiet]\u00c2\u00a0<\/b><\/p>\n<p><b>\u00c2\u00a0<\/b><\/p>\n<p><span style=\"font-size: 24px; font-weight: bold; line-height: 36px;\">Speeding up Distributed Request-Response Work\u00ef\u00ac\u201aows<\/span><\/p>\n<p>http:\/\/conferences.sigcomm.org\/sigcomm\/2013\/papers\/sigcomm\/p219.pdf<\/p>\n<ul>\n<li><span style=\"line-height: 13px;\">Latency matters ==&gt; Strict SLAs for latencies<\/span><\/li>\n<li>Why? Transient hotspots, network congestion etc.<\/li>\n<li>Web services are implemented as complex workflows with a collection of input\/output dependencies.<\/li>\n<li>Up to 150 workflow stages, degree 40, 10 steps ==&gt; stochastic latencies accumulate<\/li>\n<li>Lots of work has been done, but we care about end-to-end latency<\/li>\n<li>Reducing latency:\n<ul>\n<li>Application level techniques:\n<ol>\n<li>Reissue a request after a timeout -&gt; uses extra resources<\/li>\n<li>Terminate a request early. Use whatever results we have got before a timeout to produce results. \u00c2\u00a0-&gt;returns \u00c2\u00a0incomplete results<\/li>\n<li>Priority and pre-emption can be used. -&gt; introduces additional contention<\/li>\n<\/ol>\n<\/li>\n<li>Results in conservative parameters for all of the above.<\/li>\n<\/ul>\n<\/li>\n<li>Kwiken: Uses workflow description and logs to get an idea for latency at different stages.<\/li>\n<li>Challenges:\n<ol>\n<li>\u00c2\u00a0Different stages show different gains with different techniques<\/li>\n<li>Composing costs across stages is non-trivial<\/li>\n<li>Decomposing end-to-end latency into indivitual components is hard.<\/li>\n<\/ol>\n<\/li>\n<li>Kwiken: End-To-End Design<\/li>\n<li>Goal to minimise latency subject to cost &lt;= budget<\/li>\n<li>Solution #3 Minimize sum of variance in workflow latency instead<\/li>\n<li>Solution #2 [..]<\/li>\n<li>Solution #3 Construct per-stage variance response curves.<\/li>\n<li>Eval: Traces from 45 most used workflows at Bing, simulated work loads<\/li>\n<li>150 stage workload, with a 1% budget results in a 75% latency reduction using a combination or reissues and terminations.<\/li>\n<li>Conclusions\n<ul>\n<li>Complex workflows lead to high tail latencies<\/li>\n<li>Kiwiken is an end-to-end framework with combines many techniques.<\/li>\n<li><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h2>Leveraging Endpoint Flexibility in Data-Intensive Clusters<\/h2>\n<p><span style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; line-height: 18px;\">http:\/\/conferences.sigcomm.org\/sigcomm\/2013\/papers\/sigcomm\/p231.pdf\u00c2\u00a0<\/span><\/p>\n<ul>\n<li><span style=\"line-height: 13px;\">Communication is crucial to analytics. 33% of the time is spent in comms. <\/span><\/li>\n<li><span style=\"line-height: 13px;\">Network usage is\u00c2\u00a0imbalanced<\/span><\/li>\n<li><span style=\"line-height: 13px;\">\u00c2\u00a0What is the soruces of cross-rack traffic.\u00c2\u00a0<\/span><\/li>\n<li>54%\/40% from DFS writes.<\/li>\n<li>DFS - GFS, HDFS, Cosmos etc.\n<ul>\n<li>Many frameworks communicate via the DFS.<\/li>\n<li>\u00c2\u00a0Files ate 64MB to 1GB<\/li>\n<li>Each block is replicated 3x for FT<\/li>\n<li>2 fault domains<\/li>\n<li>Uniformly (random) placed for balanced storage<\/li>\n<\/ul>\n<\/li>\n<li>How to handle DFS flows? (A few seconds long)<\/li>\n<li>Observation: We don't care where the replicas end up.<\/li>\n<li>Sinbad - Faster writes by avoid contention during writes.<\/li>\n<li>Assumptions:\n<ul>\n<li>Link util is stable during the write duration<\/li>\n<li>All blocks are the same size.<\/li>\n<\/ul>\n<\/li>\n<li>Greedy placement minimizes average block\/file write time.<\/li>\n<li>Reality:\n<ul>\n<li>Link utilisations are reasonably stable<\/li>\n<li>Blocks are mostly (93%) the same size.<\/li>\n<\/ul>\n<\/li>\n<li>Sinbad puts the first block through the lowest loaded link.<\/li>\n<li>Master:\n<ul>\n<li>Prosodically estimates network hotspots<\/li>\n<li>takes greed online decisions<\/li>\n<li>adds hysteresis to next measurement<\/li>\n<\/ul>\n<\/li>\n<li>Evaluation\n<ul>\n<li>3000 node trace driven simulation.<\/li>\n<li>Simulation matches against a 100node EC2 deployment.<\/li>\n<\/ul>\n<\/li>\n<li>Simulation 1.39X, expr 1.26X. If using in memory storage 1.6X faster.<\/li>\n<li>What about storage balance?\n<ul>\n<li>Hotspots are uniformly distributed<\/li>\n<\/ul>\n<\/li>\n<li>Related work:\n<ul>\n<li>#1 increase capacity (bi-sec BW)<\/li>\n<li>#2 Decrease loads: Data locatalty<\/li>\n<li>#3 Balance usage<\/li>\n<\/ul>\n<\/li>\n<li>Planning to deploy Sinbad at Facebook.<\/li>\n<\/ul>\n<p><strong>Questions<\/strong><\/p>\n<p>Q: How do you asses the read\u00c2\u00a0performance? A: Assume that the file will be read by many readers.<strong> \u00c2\u00a0<\/strong><\/p>\n<p>Q: What does the name mean? A: Wanted to call it bad, then nbad, but that sounded bad, so called it Sinbad.<\/p>\n<p>Q: What is the network architecture assumption? A: We need to know the possible locations of bottlenecks.<\/p>\n<p>Q: [can't hear] A: used GFS open source using a plugin.<\/p>\n<p>Q: Can you apply this to task scheduling? A: The reason we didn't do it is the complexity is very high if you must consider receiving as well.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hello again from SIGCOMM 2013, Hong Kong, day 2. I&#8217;ll be live blogging as much of the event as I can, weather permitting&#8230;<\/p>\n","protected":false},"author":34,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[6,23],"tags":[87,79,86],"_links":{"self":[{"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/posts\/1287"}],"collection":[{"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/users\/34"}],"replies":[{"embeddable":true,"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/comments?post=1287"}],"version-history":[{"count":6,"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/posts\/1287\/revisions"}],"predecessor-version":[{"id":1288,"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/posts\/1287\/revisions\/1288"}],"wp:attachment":[{"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/media?parent=1287"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/categories?post=1287"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.syslog.cl.cam.ac.uk\/wp-json\/wp\/v2\/tags?post=1287"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}