Load Shedding for Evil Wizards

Explore load shedding mechanisms in Envoy through this 32-minute conference talk from KubeCon + CloudNativeCon Europe 2022. Dive deep into circuit breaking, rate limiting, admission control, and adaptive concurrency, understanding their implementations, use cases, and potential pitfalls. Gain insights into queuing and scheduling concepts behind these mechanisms, and learn how they can fail to protect against noisy-neighbor problems. Discover experimental results based on FBThrift work and explore potential community improvements for Envoy. Examine topics such as Little's Law applied to servers, simulating overload, rate/bandwidth limits vs. concurrency control, shortcomings of circuit breakers, and the adaptive concurrency control filter. Understand the gradient, concurrency limit behavior, configuration of adaptive concurrency, and its drawbacks, as well as the admission control filter.

Intro
What is Load?
Little's Law.
Little's Law Applied to Servers.
Avoiding Collapse
Simulating Overload
Part II: Examining the Envoy Load Shedding Mechanisms
Rate/Bandwidth Limits vs. Concurrency Control
Shortcomings of Circuit Breakers
What if I told you there was a better way?
Adaptive Concurrency Control Filter
The gradient.
Concurrency Limit Behavior.
Configuring Adaptive Concurrency
Adaptive Concurrency Drawbacks
Admission Control Filter