YoVDO

Apache Airflow with Spark for Data Engineers - Full Course

Offered By: CodeWithYu via YouTube

Tags

Apache Airflow Courses Java Courses Scala Courses Docker Courses PySpark Courses Data Engineering Courses Cluster Computing Courses ETL Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Embark on a comprehensive data engineering journey with this full course that combines Apache Airflow, Docker, Spark Clusters, Scala, Python, and Java. Create an end-to-end project, developing basic jobs in multiple programming languages, submitting them to a Spark cluster for processing, and observing live results. Learn to set up a Spark Cluster and Airflow on Docker, create Spark jobs using Python, Scala, and Java, build and compile Scala and Java jobs, and analyze cluster computation results. Gain hands-on experience in workflow automation, big data analytics, and data processing techniques essential for aspiring data engineers.

Syllabus

Introduction
Creating The Spark Cluster and Airflow on Docker
Creating Spark Job with Python
Creating Spark Job with Scala
Building and Compiling Scala Jobs
Creating Spark Job with Java
Building and Compiling Java Jobs
Cluster computation results


Taught by

CodeWithYu

Related Courses

Fundamentals of Scalable Data Science
IBM via Coursera Data Science and Engineering with Spark
Berkeley University of California via edX Master of Machine Learning and Data Science
Imperial College London via Coursera Data Analysis Using Pyspark
Coursera Project Network via Coursera Building Machine Learning Pipelines in PySpark MLlib
Coursera Project Network via Coursera