Model Serving at the Edge - Challenges and Solutions with ModelMesh
Offered By: CNCF [Cloud Native Computing Foundation] via YouTube
Course Description
Overview
Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore the challenges and solutions for deploying AI models on edge devices in this conference talk. Discover how ModelMesh, combined with K3s and MicroShift technologies, can simplify model serving at the edge. Learn about the multi-model serving backend of KServe and how it offers a small-footprint control-plane for managing model deployments on Kubernetes. Understand how ModelMesh utilizes multi-model runtimes with intelligent model loading/unloading to maximize limited resources while serving multiple models for inference. Gain insights into the generations of computing, machine learning lifecycle, complexities of model serving, and Kubernetes at the edge. Explore KServe's easy-to-use interfaces and standardized inference protocol. Dive into the ModelMesh architecture, serving runtimes, and its application on edge devices. Examine an example deployment and learn about higher density deployment challenges.
Syllabus
Intro
Outline
Generations of Computing
Machine Learning Lifecycle
Complexities of Model Serving
Kubernetes at the Edge
Introducing KServe
Easy to Use Interfaces
Kserve Standardized Inference Protocol
Enter ModelMesh
ModelMesh Architecture
ModelMesh Serving Runtimes
ModelMesh On Edge?
Example Deployment
Higher Density Deployment
Challenges
Taught by
CNCF [Cloud Native Computing Foundation]
Related Courses
A Beginner’s Guide to DockerPackt via FutureLearn A Beginner's Guide to Kubernetes for Container Orchestration
Packt via FutureLearn A Practical Guide to Amazon EKS
A Cloud Guru Advanced Networking with Kubernetes on AWS
A Cloud Guru AIOps Essentials (Autoscaling Kubernetes with Prometheus Metrics)
A Cloud Guru