DBT and Python - How to Write Reusable and Testable Pipelines

Explore how to leverage DBT (data build tool) and Python to create reusable and testable data pipelines in this 31-minute conference talk from EuroPython 2024. Discover how Flatiron Health utilizes DBT to improve cancer patient outcomes through data-driven insights. Learn about implementing version-controlled directed acyclic graphs (DAGs) with testable and reusable nodes, incorporating both SQL and Python models. Gain practical knowledge on unit and data testing techniques, as well as developing a reusable variable library. Ideal for professionals with data warehouse or data lakehouse experience, this talk demonstrates the application of software engineering best practices to data pipelines using SQL, Python, and Snowflake.

DBT & Python - How to write reusable and testable pipelines — Florian Stefan