Clipper - A Low-Latency Online Prediction Serving System

Explore a conference talk on Clipper, a low-latency online prediction serving system designed to address the challenges of deploying machine learning models in real-time applications. Learn about Clipper's modular architecture that simplifies model deployment across various frameworks and applications. Discover how the system improves prediction latency, throughput, accuracy, and robustness through techniques like caching, batching, and adaptive model selection. Examine Clipper's performance on four machine learning benchmark datasets and its comparison to TensorFlow Serving. Gain insights into how Clipper enables model composition and online learning to enhance prediction accuracy and robustness without modifying underlying machine learning frameworks.

Introduction
Machine Learning
Prediction Serving
Oneoff Systems
Architecture
Experiments
Comparison to Tensorflow
Comparison to SIF
Model abstraction layer
Multiple models per application
Wrapup