Bridging the Gap Between Batch and Real-Time with Mixed-Latency Pipelines

Explore the intricacies of building mixed-latency data platforms in this insightful 13-minute talk from Data Council. Delve into Will Goldstein's extensive experience from dynamic startups like Better to grasp time horizons, manage data freshness, and tackle pipeline bottlenecks effectively. Learn how to merge batch processing with low-latency data processing while maintaining explicit lineage and ensuring a unified view of processing jobs across different sources. Gain valuable tools and strategies to create reliable and consistent data products in mixed-latency environments.

Bridging the Gap Between Batch and Real-Time with Mixed-Latency Pipelines