Production ML Monitoring - Outliers, Drift, Explainers & Statistical Performance

Explore advanced production machine learning monitoring techniques in this 26-minute EuroPython 2021 conference talk by Alejandro Saucedo. Dive into practical examples of deploying and monitoring image classification models using TensorFlow, covering essential concepts such as concept drift, outlier detection, and explainability. Learn how to implement AI explainers, outlier detectors, concept drift detectors, and adversarial detectors as architectural patterns. Gain insights into standardized interfaces for scaling monitoring across numerous heterogeneous machine learning models. Discover best practices from data scientists, software engineers, and DevOps professionals to tackle the challenges of machine learning monitoring at scale in the Python ecosystem.

Introduction
Motivations
Overview
Anatomy of Production ML
Key Areas
Performance Monitoring
Orchestration Tools
Statistical Monitoring
Metric Servers
Outliers Drift
Explainability
Explainer
Explainer Intuition
Ensemble Patterns
Alerts
Wrap up