Building Next-Generation Data Pipelines with Containers - Fast Track to Modernization

Discover how to leverage containers for building efficient and scalable data-processing pipelines in this 25-minute conference talk from Google Cloud Next. Learn about three key use cases for Cloud Run: streaming data processing, batch jobs, and on-demand data processing. Explore the differences between push-based and pull-based data processing, and gain insights into choosing between Cloud Functions and Cloud Run. Examine a real-world example from L'Oreal to understand practical applications. Gain valuable knowledge on modernizing your data pipelines and effectively managing increasing volumes of business data using container technology.

Intro
What are containers
Ondemand data processing
Batch data processing
Streaming data processing
Pushbased vs pullbased
Should I use Cloud Functions or Cloud Run
Pullbased data processing
Pullbased data processing with Cloud Run
Example LOreal
Summary