YoVDO

Enhancing the Performance Testing Process for gRPC Model Inferencing at Scale

Offered By: CNCF [Cloud Native Computing Foundation] via YouTube

Tags

Performance Testing Courses Machine Learning Courses Kubernetes Courses Grafana Courses Prometheus Courses gRPC Courses Load Testing Courses KServe Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore the intricacies of performance testing for gRPC model inferencing at scale in this informative conference talk. Discover how to set up a Kubernetes cluster with KServe's ModelMesh for high-density deployment of machine learning models. Learn about load testing thousands of models and utilizing Prometheus and Grafana for monitoring key performance metrics. Gain insights into the complexities of model deployment, scalability challenges, and the features of Model Mesh. Delve into the automation of performance testing, including the setup of testing environments, QFlow pipeline, and K6 load tools. Witness a demonstration of the testing process, analyze testing logs and results, and understand the implications of cashmiss actions. Evaluate the benefits of using Model Mesh for your specific use case.

Syllabus

Introduction
Model Deployment
Kubernetes
Complexities
Kserve
Scalability
Model Mesh
Model Mesh Features
Performance Testing Automation
Performance Testing Setup
Performance Testing Environment
QFlow Pipeline
K6 Load Tools
GRPC
Prometheus
Demo
Testing
Testing Log
Testing Results
Cashmiss Action
Should I use Model Mesh


Taught by

CNCF [Cloud Native Computing Foundation]

Related Courses

Introduction to Artificial Intelligence
Stanford University via Udacity
Natural Language Processing
Columbia University via Coursera
Probabilistic Graphical Models 1: Representation
Stanford University via Coursera
Computer Vision: The Fundamentals
University of California, Berkeley via Coursera
Learning from Data (Introductory Machine Learning course)
California Institute of Technology via Independent