Evolving Data Pipelines at Scale with SQLMesh

Discover the innovative SQLMesh open-source project in this 30-minute conference talk from Data Council. Explore how industry giants like Airbnb, Apple, Google, and Netflix have shaped this solution to address common challenges in data pipeline development. Learn about virtual data environments, automatic change categorization, continuous testing, and accurate change previews. Gain insights into scaling data pipelines, improving data productivity, versioning data, and managing data environments. Watch a demonstration of SQLMesh's capabilities and understand concepts such as forward-only changes, semantic understanding, and handling breaking changes. Presented by Iaroslav Zeigerman, Co-Founder and Chief Architect at Tobiko Data, this talk offers valuable knowledge for data practitioners seeking more efficient and cost-effective solutions in the evolving landscape of data pipeline development.

Intro
Why am I here
Scale
Data Productivity
Data Version
Data Environment
Forward Only Change
Semantic Understanding
Breaking Changes
Demo