Ray Data Streaming for Large-Scale ML Training and Inference

Explore Ray Data streaming for large-scale machine learning training and inference in this 30-minute talk. Dive into the challenges of ML pipelines spanning CPU and GPU devices in distributed environments, focusing on batch inference and distributed training scenarios. Learn how Ray Data streaming, introduced in Ray 2.6, scales data preprocessing across heterogeneous CPU/GPU clusters. Discover practical applications through a video inference pipeline example and understand how to leverage Ray for your own ML workflows. Gain insights into building efficient data pipelines, handling record creation, and maximizing compute resource utilization. Perfect for developers and data scientists looking to optimize their ML pipelines and scale AI workloads effectively.

Introduction
What is streaming
Why did we do this
Video inference pipeline
Running the pipeline
Training
Data Streaming
Array Data Pipeline
Building a Record
Summary
Questions