Build a Multi-Tenant Training Platform Based on Kubeflow at Tencent

Explore a conference talk on building a multi-tenant training platform based on Kubeflow at Tencent. Discover how machine learning workflows at Tencent have been migrated to Kubernetes using Kubeflow and Kubeflow Pipelines, resulting in increased model development speed and improved GPU utilization. Learn about the self-serve multi-tenant platform built on Kubernetes for ML developers, utilizing Kubeflow and Virtual-Kubelet for isolated environments. Gain insights into configuring and triggering distributed machine learning jobs through API and extended kubectl command line. Understand techniques for enhancing GPU utility, including smart batch scheduling, GPU sharing, and NVidia-docker start optimization. The talk covers multi-tenancy on Kubernetes with Kubeflow, native Kubeflow federation cluster with Virtual-Kubelet, and methods to improve GPU utilization and performance. Topics discussed include Kubeflow & K8S Operator, MPI-Operator, multi-tenancy implementation, batch scheduler Volcano, job creation latency, and future developments in this area.

Intro
About Me
Our Business
Background
System Setup
Kubeflow & K8S Operator
Kubeflow: MPI-Operator
Kubeflow Muti-Tenancy Pal.
Muti-Tenant: Virtual-Kubele.
Kubeflow + Multi-Tenant
Batch Scheduler - Volcano
Job Creation Latency
Future Works