Google Cloud Real-Time Streaming - End-to-End Data Engineering Project

Build an end-to-end data engineering pipeline for real-time YouTube Analytics in this comprehensive tutorial. Learn to fetch data from the YouTube API using Python, set up a Kafka ecosystem with Docker and Confluent containers, process and stream data using ksqlDB, and send real-time notifications to Telegram. Explore advanced Python concepts and Google Cloud configuration for YouTube. Gain hands-on experience with technologies such as Python, Google Cloud, Docker, and Telegram. Follow along as the instructor guides you through setting up the system architecture, obtaining a YouTube API key, streaming data to Kafka, and connecting to external systems. By the end of this tutorial, you'll have created a powerful pipeline that notifies you instantly of any activity on your chosen YouTube videos or playlists.

Introduction
Setting up the system architecture on Docker
Control Center Demo
Getting Youtube API Key from Google Cloud
Fetching Data From Youtube with Python
Streaming Data to Kafka
Advanced Python Concept
Stream Processing with KSQLDB
Setting up Telegram Bot
Connecting to external systems from Kafka Telegram
Outro