YoVDO

Optimizing Kafka Consumers with Kubernetes and KEDA - Lessons from the Trenches

Offered By: CNCF [Cloud Native Computing Foundation] via YouTube

Tags

Kubernetes Courses Scalability Courses Real-Time Data Processing Courses Vertical Scaling Courses KEDA Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore strategies for optimizing Kafka consumer services using Kubernetes and KEDA in this informative conference talk. Learn how to enhance scalability and efficiency while maintaining service level agreements (SLAs) in real-time data processing. Discover the journey from using Horizontal Pod Autoscaler (HPA) with CPU and memory metrics to implementing KEDA's custom metrics for Kafka consumer lag. Gain insights into achieving over 75% efficiency while honoring data freshness SLAs. Delve into the implementation of a Custom Controller Resource Definition (CRD) for dynamic vertical scaling in Kafka topics, complementing KEDA's handling of daily traffic fluctuations. Understand the benefits of this holistic approach and valuable lessons learned in the pursuit of peak efficiency for Kafka consumers in a cloud-native environment.

Syllabus

Optimizing Kafka Consumers with Kubernetes and KEDA...-Shubham Badkur, Yuanzhe Liu & Simran Aggarwal


Taught by

CNCF [Cloud Native Computing Foundation]

Related Courses

Processing Real-Time Data Streams in Azure
Microsoft via edX
Gérez des flux de données temps réel
CentraleSupélec via OpenClassrooms
Data Streaming
Udacity
Taming Big Data with Apache Spark and Python - Hands On!
Udemy
Python & Cryptocurrency API: Build 5 Real World Applications
Udemy