Overload Control for µs-scale RPCs with Breakwater

Explore a conference talk on overload control for microsecond-scale Remote Procedure Calls (RPCs) using Breakwater. Learn about the challenges of maintaining high request throughputs and low tail latency in modern datacenter applications composed of numerous microservices. Discover how Breakwater, a novel server-driven admission control scheme, prevents overload in microsecond-scale services by issuing credits based on server-side queueing delay. Understand techniques such as demand speculation, piggybacking of client-side demand information, and active queue management for handling load bursts. Compare Breakwater's performance to other approaches like DAGOR and SEDA, and see how it achieves stable performance within 20 ms during unexpected demand spikes. Gain insights into the implementation details, benefits, and evaluation results of this innovative overload control scheme designed for high-performance datacenter environments.

Intro
Trend: High Fan-out Internet
Causes of Server Overload
Performance Without Overload Control
Ideal Overload Control
Strawman #1: Server-side AQM
Strawman #2: Client Rate limiting
Breakwater Overload control scheme for us-scale RPCs
Breakwater's benefits Handles server overload with us-scale RPCs with
Queueing delay as congestion signal
Credit-based admission control Breakwater controls amount of incoming requests with credits
Demand Message Overhead Server needs to know which client has demand
Impact of Credit-based Admission Control Credit-based admission control has lower and bounded tail latency but lower throughput.
Piggybacking Demand Information Breakwater piggybacks clients' demand information into requests.
Demand Speculation Breakwater speculate clients' demand to minimize message overhead
Impact of Adding Demand Speculation Demand speculation improves throughput with higher tail latency
Credit Overcommitment Server issues more credit than the number of requests it can accomodate
Incast Causing Long Queue With credit overcommitment, multiple requests may arrive at the server at the same time
Delay-based AQM To ensure low tail latency, the server drops requests if queueing delay exceeds threshold.
Impact of Adding Delay-based AQM Breakwater achieves high throughput and low and bounded tail latency at the same time
Evaluation
High Goodput with Fast Convergence
Fast Notification of Reject
Conclusion • Breakwater is a server-driven credit-based overload