YoVDO

Apache Airflow with Python

Offered By: YouTube

Tags

Apache Airflow Courses Python Courses pandas Courses REST APIs Courses

Course Description

Overview

Learn to harness the power of Apache Airflow with Python in this comprehensive tutorial. Explore essential concepts including email operators, Docker Compose integration, DAG organization, REST API triggering, user authentication, and achieving parallelism in ETL workflows. Dive into advanced topics such as Task Groups, scheduler reliability, AWS S3 log storage, and dynamic DAG addition using AWS Lambda. Master Airflow's capabilities to streamline and optimize your data pipeline management.

Syllabus

Learning Apache Airflow with Python in easy way in 40 Minutes.
How to use Apache Airflow Email Operator in docker compose file Python.
Apache Airflow + Flower + Celery Worker with Docker Compose.
organizing Dags and Helper function in Airflow Python.
Trigger Airflow Dag Via REST API tutorials.
How to correctly do Airflow User Authentication.
Learn how you can achieve parallelism when creating ETL or workflows using Pandas & Airflow.
Airflow2 docker compose with celery Learn about Task Group in Airflow.
Airflow 2: What happens if the scheduler goes down in Airflow? | Docker Compose.
Apache Airflow2: Storing logs in AWS S3 | docker compose |.
Airflow 2 Adding a new Dag Dynamically to Dags Folder with AWS S3 and Lambda.


Taught by

soumilshah1995

Related Courses

Introduction to Airflow in Python
DataCamp
Building Data Engineering Pipelines in Python
DataCamp
The Complete Hands-On Introduction to Apache Airflow
Udemy
Apache Airflow: The Hands-On Guide
Udemy
ETL and Data Pipelines with Shell, Airflow and Kafka
IBM via Coursera