Machine Learning Using Kubeflow and Kubernetes

Explore machine learning workflows on Kubernetes using Kubeflow in this 51-minute Devoxx conference talk. Learn why Kubernetes is well-suited for single- and multi-node distributed training, model training, and production inference deployment. Discover how to leverage KubeFlow and TensorFlow for machine learning needs, set up ML pipelines, and utilize visualization tools like TensorBoard for monitoring. Gain insights into distributed training with Horovod and understand Kubeflow's components, including Jupyter notebooks, TensorFlow training and inference, and hyperparameter tuning with Katib. Dive into topics such as Amazon EKS for running Kubernetes in the cloud, scaling clusters, Kubeflow requirements and deployment options, Kubeflow Fairing, and creating Kubeflow Pipeline components. Explore practical applications like consumer loan acceptance scoring and machine learning pipelines for Kubernetes on AWS.

Intro
Machine Learning is Hard!
Storage and Analytics for Machine Learning
Amazon EKS: run Kubernetes in cloud
Getting started with Amazon EKS
Set up K8s for ML: Option 1
Scaling the cluster
Kubeflow Requirements
Kubeflow on Desktop
Kubeflow on Cloud
Jupyter Notebook
Kubeflow Fairing
Hyperparameter Tuning using Katib
Katib System Architecture
Pluggable Interface
Distributed Training using Horovod
Kubeflow Pipelines
Creating Kubeflow Pipeline Components
Consumer Loan Acceptance Scoring
Machine Learning pipeline for kubernetes on AWS