From ETL to Data Engineering with Azure Data Factory

Discover how to transform your ETL processes into modern data engineering pipelines using Azure Data Factory in this comprehensive conference talk. Learn about scalability, successful patterns, and modeling common scenarios as you progress from an ADF novice to an expert pipeline creator. Explore the new features in ADFV2 and their application in streamlining factories within real-world solutions. Gain insights into compute options provided by ADF Mapping Data Flows and their interaction with Azure Databricks. Cover essential topics including Linked Services, Development Tools, Integration Runtimes, parameter usage, and Mapping Dataflows. Witness a live demo on transformations and compare Data Factory with Logic Apps. By the end of this session, you'll be equipped with the knowledge to implement truly cloud-native ETL solutions in your data platform.

Intro
What is Data Factory
Linked Services
Development Tools
Integration Runtimes
Data Factory
Using Parameters
Databricks
Mapping Dataflow
Dataflow Preview
Audience Questions
Demo
Transformations on the fly
Data Factory vs Logic Apps
Conclusion