From Alert Notification to Comparison of Good and Bad Requests in One Click

Explore a 24-minute conference talk from the Cloud Native Computing Foundation (CNCF) that demonstrates how to streamline the process from alert notification to comparing good and bad requests in a single click. Learn how to leverage open-source tools like Prometheus, Jaeger, Grafana, and M3 to reduce time-to-mitigation by providing relevant context for troubleshooting. Discover recent advances in the community that enable automatic jumping from metrics-based alerts to representative problematic traces, and compare them with non-problematic ones. Gain insights into the integration of metrics, logs, and traces, as well as the implementation of OpenTelemetry, exemplars, and contextual links to enhance your observability workflow. Understand the complete ingestion pipeline and explore resources to further improve your on-call experience and incident response capabilities.

Intro
Agenda
Metrics.
Logs..
Traces..
On-call Experience Today
Can we jump there automatically?
Tracing and Metrics
We can actually jump to the trace directly...
Open Metrics and Exemplars
OpenTelemetry: Instrumentation SDK
Prometheus / M3
Trace Sampling
A Complete Ingestion Pipeline
How?
M3 Query and Exemplars
Selecting a good/bad source for traces?
Building Contextual Links
Summary
Where are we on this journey?
Resources
Thank you and Q&A