YoVDO

Building Realtime Data Warehouses from Scratch - End-to-End Data Engineering Project

Offered By: CodeWithYu via YouTube

Tags

Data Warehousing Courses Data Visualization Courses Apache Airflow Courses Apache Kafka Courses Data Engineering Courses Real-Time Analytics Courses ETL Courses Dimensional Modeling Courses Apache Pinot Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Embark on a comprehensive journey to build a real-time data warehouse from scratch in this 2-hour video tutorial. Design and implement a complex real-time data warehouse architecture, set up Apache Airflow, Kafka, and Apache Pinot for seamless data pipelines, and develop custom Apache Airflow hooks for Kafka & Pinot integration. Learn to ingest batch and streaming data into Apache Pinot for real-time analytics, create a dynamic dashboard with Apache Superset to visualize evolving data in real-time, and apply dimensional modeling for better data organization and reporting. Follow along as the instructor guides you through each step, from system architecture and project setup to creating dimensional models, connecting various components, and finally setting up a real-time dashboard. Gain practical experience in big data engineering, ETL processes, and data analytics while building a complete end-to-end data engineering project.

Syllabus

Introduction
System Architecture
Setting up the project
Creating Dimensional Modelling with Apache Airflow
Creating Apache Airflow Hook for Kafka
Creating Apache Airflow Hook for Apache Pinot
Connecting Apache Pinot to Kafka
Batch Data Ingestion for Apache Pinot
Setting up Apache Superset for Data Visualisation
Creating Superset Dataset for Visualisation
Creating Apache Superset Realtime DW Dashboard
Wrapping up
Outro


Taught by

CodeWithYu

Related Courses

内存数据库管理
openHPI
CS115x: Advanced Apache Spark for Data Science and Data Engineering
University of California, Berkeley via edX
Processing Big Data with Azure Data Lake Analytics
Microsoft via edX
Google Cloud Big Data and Machine Learning Fundamentals en Español
Google Cloud via Coursera
Google Cloud Big Data and Machine Learning Fundamentals 日本語版
Google Cloud via Coursera