Praemonitus Praemunitus - Using Actionable Alerts to Fix Problems

Discover how to create and implement actionable alerts for efficient problem-solving in this 24-minute conference talk from Conf42 O11y 2024. Learn the key components of effective alerts, including when to alert, what information to capture, and how to deliver notifications. Explore the concept of "praemonitus praemunitus" (forewarned is forearmed) and understand the importance of automating responses when possible. Gain insights into determining whether an alert is truly actionable, if its cause is deterministic, and when human intervention is necessary. Watch a practical demonstration of automating alerts in Google Cloud using Cloud Monitoring Alerting, covering service and resource selection, timing, and alert content. By the end of this talk, you'll be equipped with strategies to optimize your alerting system and improve your team's response to potential issues.

intro
preamble
nice to meet you
agenda
praemonitus praemunitus forewarned is forearmed
components of efficient alert
when to alert
what to capture about alert
how to alert
don't send an engineer to do a machine's job
automate, ticket, or page?
components of automated alert
demo time: automate alert in google cloud
cloud monitoring alerting
- service and resource
- when
- what and how
is alert actionable?
is cause deterministic?
does response require human intervention?
wrapping up
thank you!