YoVDO

Serving Machine Learning Models at Scale Using KServe

Offered By: CNCF [Cloud Native Computing Foundation] via YouTube

Tags

Conference Talks Courses Kubernetes Courses Scalability Courses Serverless Computing Courses Model Development Courses Performance Testing Courses KServe Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore the scalable deployment of machine learning models using KServe in this conference talk. Learn about the Multi-Model Serving solution designed to address limitations in the 'one model, one service' paradigm, including resource constraints, pod limitations, and IP address restrictions. Discover how KServe enables efficient GPU utilization for multiple models, and gain insights into its components, standard inference protocols, and performance benchmarks. Understand the evolution from KFServing to KServe, the challenges in model development, and the roadmap for future improvements. Dive into the design of Multi-Model Serving and its implementation across different frameworks, showcasing its potential to revolutionize machine learning model deployment at scale.

Syllabus

Introduction
Background about KServe
Milestones
Model Development
Challenges
KServe
KServe Components
Standard Inference Protocol
HTTP Protocol
GRPC Protocol
New Scalability Problem
Current Approach
Problem
Compute resource limitations
Maximum pod limitations
Maximum IP address limitations
Model Mesh Solution
Performance Test
Latency Test
Model Mesh
Roadmap
Questions
Original Design


Taught by

CNCF [Cloud Native Computing Foundation]

Related Courses

Data Science in Action - Building a Predictive Churn Model
SAP Learning
Applied Data Science Capstone
IBM via Coursera
Data Modeling and Regression Analysis in Business
University of Illinois at Urbana-Champaign via Coursera
Introduction to Predictive Analytics using Python
University of Edinburgh via edX
Machine Learning con Python. Nivel intermedio
Coursera Project Network via Coursera