YoVDO

Apache Airflow with Python

Offered By: YouTube

Tags

Apache Airflow Courses Python Courses pandas Courses REST APIs Courses

Course Description

Overview

Learn to harness the power of Apache Airflow with Python in this comprehensive tutorial. Explore essential concepts including email operators, Docker Compose integration, DAG organization, REST API triggering, user authentication, and achieving parallelism in ETL workflows. Dive into advanced topics such as Task Groups, scheduler reliability, AWS S3 log storage, and dynamic DAG addition using AWS Lambda. Master Airflow's capabilities to streamline and optimize your data pipeline management.

Syllabus

Learning Apache Airflow with Python in easy way in 40 Minutes.
How to use Apache Airflow Email Operator in docker compose file Python.
Apache Airflow + Flower + Celery Worker with Docker Compose.
organizing Dags and Helper function in Airflow Python.
Trigger Airflow Dag Via REST API tutorials.
How to correctly do Airflow User Authentication.
Learn how you can achieve parallelism when creating ETL or workflows using Pandas & Airflow.
Airflow2 docker compose with celery Learn about Task Group in Airflow.
Airflow 2: What happens if the scheduler goes down in Airflow? | Docker Compose.
Apache Airflow2: Storing logs in AWS S3 | docker compose |.
Airflow 2 Adding a new Dag Dynamically to Dags Folder with AWS S3 and Lambda.


Taught by

soumilshah1995

Related Courses

Amazon SageMaker: Build an Object Detection Model Using Images Labeled with Ground Truth (Vietnamese)
Amazon Web Services via AWS Skill Builder
Amazon SageMaker: Build an Object Detection Model Using Images Labeled with Ground Truth (German)
Amazon Web Services via AWS Skill Builder
Amazon SageMaker: Build an Object Detection Model Using Images Labeled with Ground Truth (Korean)
Amazon Web Services via AWS Skill Builder
Amazon SageMaker: Build an Object Detection Model Using Images Labeled with Ground Truth (Simplified Chinese)
Amazon Web Services via AWS Skill Builder
Amazon SageMaker: Build an Object Detection Model Using Images Labeled with Ground Truth (French)
Amazon Web Services via AWS Skill Builder