Scaling Data Reliably: A Journey in Growing Through Data Pain Points

Explore the challenges and solutions in scaling data systems reliably in this 16-minute conference talk by Miriah Peterson at MLOps.community. Delve into the concept of Data Downtime and its impact on business outcomes. Learn how Data Reliability Engineering (DRE) teams adapt Site Reliability Engineering (SRE) practices to enhance data system reliability. Discover key strategies for strengthening data pipelines, including minimizing Data Downtime, implementing Data Service Level Metrics, and effective data monitoring techniques. Gain insights into distinguishing between system failures and data failures, and understand how to leverage metadata for preventative data engineering practices. Drawing from real-world scenarios, this talk provides practical knowledge for data professionals aiming to build more robust and reliable data systems.

Scaling Data Reliably: A Journey in Growing Through Data Pain Points // Miriah Peterson // DE4AI