Scheduling Framework: Batch Extensions with Apache YuniKorn

Explore batch extensions for Kubernetes scheduling using Apache YuniKorn in this informative conference talk. Delve into the challenges of running batch and data processing workloads on Kubernetes, and discover how Apache YuniKorn extends the scheduler to address these issues. Learn about job-aware preemption, gang scheduling, and queueing with quota enforcement while scheduling. Understand how these opt-in features can be used alongside existing default functionality to improve flexibility and efficiency in handling diverse workload types, including Apache Spark jobs. Gain insights into managing dynamic job requirements, handling pod creation on the fly, and mitigating issues related to quota enforcement and management pod preemption.

Scheduling Framework: Batch Extensions with Apache YuniKorn - Wilfred Spiegelenburg & Peter Bacsko