Propagating Programming Paradigms - Lifting High Performance Compute Into Kubernetes

Explore how to integrate High Performance Computing (HPC) paradigms into Kubernetes clusters to optimize performance and sustainability without increasing user complexity. This conference talk delves into the similarities between AI/ML and traditional HPC workloads, highlighting the challenges users face when deploying in HPC environments. Learn about the history of HPC's shortcomings in hardware abstraction and discover current Kubernetes projects that enhance performance in a cloud-native manner. Gain insights into improving Kubernetes to maximize both performance and sustainability while maintaining user-friendly deployment processes. Understand how to build systems where hardware, network, storage, and security engineers can focus on leveraging underlying hardware for optimal performance, allowing users to concentrate solely on their workloads' execution and completion times.

Propagating Programming Paradigms: Lifting High Performance Compute Into K8s - Marlow Weston , Intel