The Challenges and Opportunities of Continual Learning in Real-Time Machine Learning

Explore the challenges and opportunities of continual learning in machine learning ecosystems in this 25-minute conference talk from the 2022 The Future of Data-Centric AI conference. Delve into the four stages of continual learning, compare stateful and stateless training, and examine key challenges in the field. Discover solutions for feature monitoring and evaluation, and gain insights into batch prediction versus online prediction, train-predict inconsistency, and deployment strategies. Learn about smart triggers for retraining, fresh data challenges, and the importance of real-time monitoring. Understand temporal shifts and their impact on time window scales, and explore the complexities of monitoring features in continual learning systems.

Claypot
Batch prediction vs. online prediction
Online prediction with batch features
Online prediction with online features
Train-predict inconsistency
"Easy" deployment: static
"Hard" deployment: continual
4 stages of continual learning
Smart triggers for retraining
Continual deployment challenges
Fresh data challenge
Algorithm challenge
Evaluation challenge
Real-time monitoring vs. batch monitoring
What to monitor
Temporal shifts: time window scale matters
Monitoring features: challenges
Monitoring solutions