Declarative MLOps: Streamlining Model Serving on Kubernetes

Explore a comprehensive 59-minute conference talk on declarative MLOps and streamlining model serving on Kubernetes. Discover how to leverage native K8s operators for deploying models, learn best practices for containerizing models, and implement CI/CD using GitOps. Gain insights into three ways of serving ML models: HTTP endpoints, message queues, and batch processing. Dive into the process of building custom containers, creating predictors, and implementing continuous integration and delivery workflows. Learn about tools like Github Actions, Argo CD, and Kustomize for efficient MLOps practices. Understand the benefits of the declarative paradigm in MLOps, including scalability and compatibility with various tools. Perfect for data scientists and ML engineers looking to enhance their model deployment and management skills in production environments.

[] Musical introduction to Rahul Parundekar
[] LLMs in Production Conference announcement
[] Purchase our Swag shirt!
[] Declarative Paradigm
[] Why now?
[] It's great for scalability
[] Most MLOps tools work well with K8s
[] Easy-deploys with tool-provided CRDs
[] Caveats
[] This talk
[] 3 Ways to Serve ML Models
[] Way 1: Serving a Model with an HTTP Endpoint
[] Way 2: Serving the Model with a Message Queue
[] Way 3: Long-running Task that Performs Batch Processing
[] Buil your own container
[] The main predictor 1/2: Singleton with load method
[] The main predictor 2/2: Predict
[] Way 1 5 steps
[] Way 2 2 steps
[] Way 3 2 steps
[] Tests: Sanity check for the model
[] Bringing it together: Entrypoint
[] Continuous Integration CI
[] Create docker-compose.yaml to make it easier for CI
[] On PR: Run tests with Github Actions
[] Branch-protection
[] On PR: Github Actions automatically runs our test
[] On PR: PRs can be then merged on approval
[] Container Repository
[] Continuous Integration CI
[] On merge to main
[] Actions that can constraint
[] TODO
[] Continuous Delivery
[] Argo CD
[] Image promotion with Kustomize
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]