How to Trade off Server Utilization and Tail Latency

Explore a 28-minute conference talk from SREcon19 Asia/Pacific that delves into the trade-offs between server utilization and tail latency in large-scale systems. Learn about the fundamentals of queueing theory and its practical applications in system performance optimization. Discover how increasing average utilization impacts tail latency, and gain insights into measuring and analyzing these crucial metrics. Follow along with a schematic diagram, request timeline, and service time explanations to better understand the concepts. Acquire valuable takeaways and basic rules for balancing utilization and tail latency in your own systems. Presented by Julius Plenz from Google, this talk offers a concise yet comprehensive overview of this important topic in site reliability engineering.

Introduction
Schematic diagram
High utilization vs low latency
Common problem with utilization
Request timeline
Service time
Measuring tail latency
Measuring utilization
Queueing effects
Takeaway
Conclusion