Roles and Responsibilities of the Azure Data Engineer

Explore the evolving role of Azure Data Engineers in this comprehensive conference talk from PASS Data Community Summit. Discover the key responsibilities, essential skills, and Azure services crucial for aspiring data engineers. Learn about the data science hierarchy of needs, data engineering pillars, and the differences between ETL and ELT processes. Gain insights into DevOps practices, scripting languages, automation, and scaling techniques. Understand the importance of infrastructure as code and familiarize yourself with various Azure services, including databases, data stores, Azure Data Factory, and Azure Stream Analytics. Align your career path with the Microsoft Azure Data Engineer Associate certification and position yourself for success in this rapidly growing field.

Intro
Housekeeping
Community
Session Evaluation
Who am I
Learning Pathways
Agenda
Why Data Engineer
Data Engineering is a discipline
Data Science Hierarchy of Needs
Data Engineering Pillars
Infrastructure
Data ingestion
Data pipelines
ETL vs ETL
Extract Transform Load
Data Modeling
Skills
Understanding DevOps
Scripting Languages
Automation
Scaling
Infrastructure as Code
Azure Data Engineer Certification
Databases
Other Data Stores
Azure Data Factory
Azure Stream Analytics
Data Analytics