Using Kubernetes for Machine Learning Frameworks

Explore how to leverage Kubernetes for machine learning frameworks in this comprehensive conference talk from GOTO Chicago 2019. Discover why Kubernetes is well-suited for training and running machine learning models in production, with a focus on setting up open-source frameworks like TensorFlow, Apache MXNet, and PyTorch on a Kubernetes cluster. Learn about the isolation, auto-scaling, load balancing, flexibility, and GPU support that Kubernetes provides for computationally intensive and data-heavy machine learning models. Dive into the training, massaging, and inference phases of setting up a machine learning framework on Kubernetes, and gain insights into the Amazon ML stack and Amazon EKS for running Kubernetes in the cloud. Explore practical examples using the MNIST database and Fashion MNIST, and understand the challenges and solutions for setting up containers for machine learning. Get acquainted with AWS deep learning containers and learn how to create a machine learning pipeline using Kubernetes and SageMaker.

Intro
Centerpiece for digital transformation
Machine Learning 101
The Amazon ML stack: Broadest & deepest set of capabilities
Amazon EKS-run Kubernetes in cloud
Amazon EKS deployment
Getting started with Amazon EKS
Creating an EKS cluster using eksctl
GPUs for Machine Learning training • Training maps to matrix multiplications • Coupled with extremely high memory bandwidth
Set up K8s for ML-option 1
Create K8s cluster for ML-option 1
Scaling the cluster
Create K8s cluster for ML-option 2
Set up K8s for ML-option 2b
Challenges in setting up containers for ML
AWS deep learning containers
16 container images
ML on K8s—without KubeFlow
MNIST database
Fashion MNIST
AWS is the platform of choice to run TensorFlow
Machine Learning using TensorFlow on K8s
Apache MXNet
Distributed training using Horovod
Machine Learning pipeline for K8s
Machine Learning pipeline using SageMaker