Real Time Stream Processing for Machine Learning at Massive Scale

Explore real-time stream processing for machine learning at massive scale in this EuroPython 2020 conference talk. Gain practical insights on building scalable data streaming ML pipelines to process large datasets using Python and popular frameworks like Kafka, SpaCy, and Seldon. Follow a case study on automated content moderation of Reddit comments, handling stream data in a Kubernetes cluster. Dive into fundamental stream processing concepts such as windows, watermarking, and checkpointing. Learn to build, deploy, and monitor complex data streaming pipelines that process production incoming data in real-time. Discover best practices and tools for monitoring, as well as an overview of the machine learning workflow, including model training, evaluation, and serving with native Kafka integration.

Intro
Hello, my name is Alejandro
The Institute for Ethical Al & Machine Learning
A trip to the past present: ETL
Specialised Tools
Batch VS-AND Streaming
Unifying Worlds
Streaming Concepts: Window
Streaming Concepts: Checkpoir.
Some Stream Processing Tools
Machine Learning Workflow
Model Training
More on EDA & Model Evaluatic
ML Stream Processing Step
ML Model Request Step
Overview of Seldon Model Servi.
Native Integration w Kafka