TSDB Data Compactions on Scale Without Headache - Now & Future

Explore the intricacies of TSDB data compactions at scale in this informative conference talk. Delve into the workings of compaction in TSDB and TSDB-based compactors with object storage. Learn about the challenges of large-scale compaction operations and strategies to mitigate issues. Discover recent improvements and new tools for rewriting TSDB blocks. Gain insights into future developments planned for enhancing compactions in the Prometheus ecosystem. Examine topics such as vertical compaction, query blocks, tombstones, planning, Thanos Compactor, Prometheus HA deployment, duplicate data, deduplication labels, custom vertical chunks merger, block rewrite, and potential extensions to TSDB compaction.

Intro
Definition
TSDB Block
meta.json
Index & Chunks
Two Blocks
Compacted Block
Vertical Compaction
Query Blocks
Tombstones
Planning
Thanos Compactor
Prometheus HA Deployment
Duplicate Data
Deduplication Label
Custom Vertical Chunks Merger
Block Rewrite
Relabel Block
Extending TSDB Compaction?
Compaction with modifiers
Merge Blocks With Modifiers
Brainstorming!