Optimizing Kafka Consumers with Kubernetes and KEDA - Lessons from the Trenches
Offered By: CNCF [Cloud Native Computing Foundation] via YouTube
Course Description
Overview
Explore strategies for optimizing Kafka consumer services using Kubernetes and KEDA in this informative conference talk. Learn how to enhance scalability and efficiency while maintaining service level agreements (SLAs) in real-time data processing. Discover the journey from using Horizontal Pod Autoscaler (HPA) with CPU and memory metrics to implementing KEDA's custom metrics for Kafka consumer lag. Gain insights into achieving over 75% efficiency while honoring data freshness SLAs. Delve into the implementation of a Custom Controller Resource Definition (CRD) for dynamic vertical scaling in Kafka topics, complementing KEDA's handling of daily traffic fluctuations. Understand the benefits of this holistic approach and valuable lessons learned in the pursuit of peak efficiency for Kafka consumers in a cloud-native environment.
Syllabus
Optimizing Kafka Consumers with Kubernetes and KEDA...-Shubham Badkur, Yuanzhe Liu & Simran Aggarwal
Taught by
CNCF [Cloud Native Computing Foundation]
Related Courses
Processing Real-Time Data Streams in AzureMicrosoft via edX Gérez des flux de données temps réel
CentraleSupélec via OpenClassrooms Data Streaming
Udacity Taming Big Data with Apache Spark and Python - Hands On!
Udemy Python & Cryptocurrency API: Build 5 Real World Applications
Udemy