Advanced Resource Management for Running AI/ML Workloads with Kueue

Explore advanced resource management techniques for AI/ML workloads using Kueue in this informative conference talk. Dive into Kueue's architecture and learn how to implement quota- and priority-based resource sharing among multiple teams on Kubernetes. Understand the decision-making process behind Kueue's scheduler for starting and stopping jobs. Gain insights from a real-world production use case at CyberAgent, where Kueue serves as a crucial component in a multi-tenant system supporting various engineers and ML research teams. Discover how Kueue manages different job types and ML frameworks across multiple CPU and GPU configurations. Address the challenge of running ML training jobs requiring all pods to be scheduled, and explore solutions using Kueue in both static and autoscaling environments with the new ProvisioningRequest API.

Advanced Resource Management for Running AI/ML Workloads with Kueue - Michał Woźniak, Yuki Iwai