How GitLab.com Uses Long-Term Monitoring Data for Capacity Forecasting

Explore how GitLab.com leverages long-term monitoring data for capacity forecasting in this insightful conference talk. Dive into the world of Tamland, a capacity planning tool built by GitLab, and discover its integration with Thanos for extensive metric storage capabilities. Learn about the predictive forecast model used to anticipate growth trends across numerous saturation points. Gain practical knowledge on capturing long-term metrics data scalably, utilizing Facebook's Prophet library for forecast modeling, and integrating with Jupyter for comprehensive report generation. Examine the benefits of adopting a data-driven, repeatable approach to capacity planning and understand the challenges faced during tool development. Delve into topics such as Postgres saturation events, system capacity planning, load testing, resource capacity modeling, and consistent utilization measurement through Prometheus recording rules. Investigate the use of Thanos for long-term metrics storage and explore initial forecasting attempts using linear regression. As an open-source project, Tamland offers attendees the opportunity to further explore its implementation.

Intro
A series of Postgres saturation events..
Definitions: Resources
Saturation Events vs. Mitigation Time
System Capacity Planning
Load Testing Maximum Capacity
Guessing Maximum Capacity
Modelling Maximum Capacity
Estimating System Capacity is Hard
Resource Capacity, not System Capacity
Resource/Service Matrix
Measuring Utilization Consistently
Prometheus Recording Rules
Aggregating to Reduce Data
Real-time Utilization Monitoring (Detail)
Using Thanos for Long-Term Metrics Storage
First Attempt at Forecasting: Linear Regression
Runaway Session State Bug