Caravan - Practical Online Learning of In-Network ML Models with Labeling Agents

Explore a conference talk on CARAVAN, a practical online learning system for in-network machine learning models. Discover how this innovative approach tackles the challenges of automatic labeling for evolving traffic and efficient monitoring of model performance degradation. Learn about CARAVAN's strategy of repurposing existing systems as high-quality labeling sources and its introduction of the accuracy proxy metric for tracking model degradation. Gain insights into how CARAVAN enables in-network ML models to adapt to changing traffic dynamics, resulting in significant improvements in F1 scores compared to offline models. Understand how this system maintains comparable inference accuracy to continuous-learning systems while substantially reducing GPU compute time through its accuracy proxy and retraining triggers.

OSDI '24 - Caravan: Practical Online Learning of In-Network ML Models with Labeling Agents