Chidori - AI/ML Cluster Management Platform

Explore the challenges and solutions of running Apache Spark on Kubernetes in this conference talk from Conf42 Cloud Native 2024. Discover the advantages of this approach and learn about Chidori, an AI/ML cluster management platform designed to address common issues. Gain insights into the high-level design of Chidori and its role in optimizing AI/ML workloads on Kubernetes. Watch a demonstration of the platform in action and understand how it can improve cluster management for data-intensive applications.

intro
preamble
agenda
spark on kubernetes
advantages
challenges
chidori
high level design
ai/ml cluster management
demo
thank you!