How and Why Prometheus' New Storage Engine Pushes the Limits of Time Series Databases

Explore the cutting-edge advancements in Prometheus' new storage engine designed to handle the challenges of modern, dynamic environments. Dive into the architecture, data structures, and performance improvements that enable this time series database to efficiently manage high turnover rates of monitoring targets and deliver consistent query performance. Learn about the evolution from version 1.0 to 2.0, including solutions to previous problems, optimized data layout, and enhanced indexing. Examine benchmarks showcasing improvements in CPU usage, disk writes, on-disk size, and compactions. Discover new features such as granular deletions, tombstone removal, and improved retention and backup capabilities. Gain insights into how this innovative storage engine pushes the limits of time series databases, making it well-suited for monitoring containerized and orchestrated environments like Docker Swarm and Kubernetes.

Intro
Time Series
Scale
Compression - Timestamp
Compression - Value
Churn!
The new era
1.0 Problems
Data and Queries
2.0 Layout
2.0 Index
Benchmarks
CPU: Cores Used
Disk Writes: MB
On Disk Size: GB
Compactions
Retention
Backups!
Granular deletes
Tombstones Removal
Try it!!!!
Questions?