Exactly-Once Semantics Revisited: Distributed Transactions Across Flink and Kafka
Offered By: Confluent via YouTube
Course Description
Overview
Explore the challenges and upcoming improvements in Exactly-Once Semantics (EOS) integration between Apache Flink and Apache Kafka in this 31-minute talk. Dive into the current pitfalls of Flink's EOS implementation when writing to Kafka, including the use of Java reflection hacks and potential data loss scenarios. Learn about the collaborative efforts of the Flink and Kafka communities to address long-standing technical debts. Gain insights into the basics of Flink's EOS achievement with external systems and common hurdles in implementing distributed transactions. Discover the proposed changes to both the Kafka transaction protocol and Flink transaction coordination aimed at creating a more robust integration. By the end of this session, understand the unique challenges of EOS with Flink and Kafka, and get a glimpse of the anticipated improvements across both projects.
Syllabus
Exactly-Once Semantics Revisited: Distributed Transactions across Flink and Kafka
Taught by
Confluent
Related Courses
Deploying Apache Pulsar to Google Kubernetes EnginePluralsight Stream Processing Design Patterns with Kafka Streams
LinkedIn Learning Apache Kafka Series - Confluent Schema Registry & REST Proxy
Udemy Apache Kafka Series - Kafka Connect Hands-on Learning
Udemy The Complete Apache Kafka Practical Guide
Udemy