hostCC
A CC Architecture for Host Congestion
hostCC is a congestion control architecture that efficiently handles host congestion, along with network fabric congestion. hostCC provides high network utilization and low latencies in the presence of bottlenecks at the host and/or the network fabric. The key idea in hostCC is using a host-local congestion response operating at a sub-RTT granularity, coupled with network congestion response at RTT granularity.
What is Host Congestion?
- Host congestion problem. Contention of resources within the host interconnect (like memory bandwidth), leading to PCIe underutilization, NIC queue build-up (and possible packet drops) and network underutilization
- Origin and trends. Driven by recent adoption of multi-gigabit access link bandwidths, and relatively stagnant technology trends for host resources. Current trends suggest problem is going to get worse with time.
- Impact. Our HotNets'22 paper characterizes the problem in Google's large-scale production clusters. Our SIGCOMM'23 paper reproduces host congestion phenomenon using open-sourced stacks and CC protocols. We observed >50% throughput degradation, orders of magnitude of latency inflation and >1% packet drop rates at the host.
Root cause of host interconnect congestion.
hostCC: Key Design Ideas
High-level overview of hostCC architecture.
hostCC embodies
three key ideas:
- Collecting host congestion signals: hostCC collects congestion signals at the host-interconnect at sub-us granularity. This allows hostCC to capture the precise time, location, and reason for host congestion.
- Sub-RTT level host-local congestion response hostCC controls the resources allocated to intra-host traffic (originating from CPUs and other peripheral devices) competing with the network traffic, at a sub-RTT granularity.
- Network congestion response using both host and network congestion signals: hostCC performs network resource allocation based on end-to-end bottleneck capacity, also including host-interconnect in addition to each network hop.
hostCC Benefits
No HW/SW modifications
Improved utilization & latency
Interpolates well with network CC
Download
hostCC code and instructions to repoduce its results available at hostCC github repo.
Contribute to hostCC. hostCC architecture does not dictate specific implementations for each of its key design decisions. Feel free to extend hostCC to incorporate (1) additional host congestion signals, (2) additional host resource allocation policies and (3) trying hostCC over additional network CC protocols.
Papers
Team Members
Feel free to reach out via email for any questions.