DevOps to MLOps: Scaling ML Models to 2 Million+ Requests per Day

Explore the journey from DevOps to MLOps in this conference talk that delves into scaling machine learning models to handle over 2 million requests per day. Learn about the key steps in MLOps, including model development, deployment, and monitoring. Discover a real-world case study on eKYC SaaS APIs, examining the cloud-agnostic architecture and scaling strategies employed. Gain insights into eliminating single points of failure, capacity planning, and cost optimization through autoscaling. Analyze production issues related to GPU utilization and high latency, and extract valuable lessons for implementing MLOps at scale. Enhance your understanding of the challenges and solutions in transitioning from traditional DevOps to MLOps for large-scale machine learning applications.

intro
preamble
chinmay naik
agenda
what is mlops
mlops steps
simpelst mlops flow
production work ahead
case study - ekyc saas apis
ml model apis
architecture
ekyc saas apis - requirements
cloud agnostic architecture
why cloud agnostic?
scaling journey
eliminate single points of failure
capacity planning
cost optimization and autoscaling
production issue 1 - gpu utilization in nomad
production issue 2 - high latency issue
lessons
keep learning