YoVDO

Model Serving at the Edge - Challenges and Solutions with ModelMesh

Offered By: CNCF [Cloud Native Computing Foundation] via YouTube

Tags

Edge Computing Courses Machine Learning Courses Kubernetes Courses K3s Courses KServe Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore the challenges and solutions for deploying AI models on edge devices in this conference talk. Discover how ModelMesh, combined with K3s and MicroShift technologies, can simplify model serving at the edge. Learn about the multi-model serving backend of KServe and how it offers a small-footprint control-plane for managing model deployments on Kubernetes. Understand how ModelMesh utilizes multi-model runtimes with intelligent model loading/unloading to maximize limited resources while serving multiple models for inference. Gain insights into the generations of computing, machine learning lifecycle, complexities of model serving, and Kubernetes at the edge. Explore KServe's easy-to-use interfaces and standardized inference protocol. Dive into the ModelMesh architecture, serving runtimes, and its application on edge devices. Examine an example deployment and learn about higher density deployment challenges.

Syllabus

Intro
Outline
Generations of Computing
Machine Learning Lifecycle
Complexities of Model Serving
Kubernetes at the Edge
Introducing KServe
Easy to Use Interfaces
Kserve Standardized Inference Protocol
Enter ModelMesh
ModelMesh Architecture
ModelMesh Serving Runtimes
ModelMesh On Edge?
Example Deployment
Higher Density Deployment
Challenges


Taught by

CNCF [Cloud Native Computing Foundation]

Related Courses

A Beginner’s Guide to Docker
Packt via FutureLearn
A Beginner's Guide to Kubernetes for Container Orchestration
Packt via FutureLearn
A Practical Guide to Amazon EKS
A Cloud Guru
Advanced Networking with Kubernetes on AWS
A Cloud Guru
AIOps Essentials (Autoscaling Kubernetes with Prometheus Metrics)
A Cloud Guru