Evolution of Real-time User Engagement Event Consumption at Pinterest - Current 2023

Explore a 34-minute conference talk from Current 2023 detailing Pinterest's evolution in real-time user engagement event consumption. Discover how Pinterest handles billions of daily user engagement events consumed by hundreds of real-time applications through Apache Kafka® clusters. Learn how Lu Liu and Heng Zhang implemented Apache Flink® to standardize processing logic and enhance data quality, resulting in multi-million dollar infrastructure savings and increased user engagement. Gain insights into Pinterest's event processing architecture, its evolution, problem-solving approaches, and the implementation of a Platinum Event Stream. Delve into the technical architecture, including the Flink application, splitter, filtering, and splitting processes. Understand key learnings, improvements in data quality, cost efficiency measures, and ongoing efforts in this comprehensive exploration of Pinterest's data infrastructure advancements.

- Intro
- What is Pinterest
- Event Processing Architecture
- Architecture Evolution
- Problem Solving
- Platinum Event Stream
- Technical Architecture
- Flink Application
- Splitter
- Filtering
- Splitting
- Learnings
- Data Quality
- Cost Efficiency
- Conclusion
- Ongoing Efforts