Patterns of ML Models in Production

Explore the journey of deploying machine learning models in production through this 26-minute PyCon US talk by Simon Mo. Learn about common deployment patterns for online serving and offline processing, backed by concrete use cases drawn from over 100 user interviews for Ray and Ray Serve. Discover architectural patterns for batch processing, ensemble models, and online learning. Understand key considerations like where to run business logic and how to implement computer vision pipelines. Gain insights into Ray Serve, a scalable model serving framework, and its deployment patterns for handling multiple models in production environments. Access accompanying slides for visual references and deeper understanding of the concepts presented.

Intro
Putting ML Models in Production
Architectural Patterns Batch
A Typical Computer Vision Pipeline
Pipeline Implementation
Ensemble Use Cases
Ensemble Deployment
Business Logic in Action
Key Question: Where to Run the Business Logic?
Online Learning
Ray Serve: Deployment
Ray Serve: Handle
Ray Serve: Patterns
Business Logic in Ray Serve
Ray Serve: A Framework for 1+ Models in Production