Delta Live Tables - Building Reliable ETL Pipelines with Azure Databricks

Learn how to build fast, reliable, scalable, and declarative ETL pipelines using Delta Live Tables on the Azure Databricks platform in this 49-minute conference talk by Mohit Batra at SQLBits. Explore the Delta Architecture Layers, understand the complexities in ETL pipelines, and discover how Delta Live Tables can simplify pipeline creation. Watch demonstrations of both simple and complex streaming pipelines, gaining practical insights into implementing these techniques. Ideal for professionals working with Azure, Spark, Data Lakes, and Big Data analytics, this session covers essential topics in data engineering and efficient data loading patterns.

Intro
Mohit Batra
Delta Architecture Layers
Complexity in ETL Pipelines
Pipelines with Delta Live Tables
Components
Demo 1: Simple Pipeline
Demo 2: Complex & Streaming Pipeline
Session Evaluation