Kafka in the Wild

Explore how to optimize Kafka data structures for reuse across projects in this conference talk from GOTOpia Chicago 2021. Learn about the challenges of maintaining consistent data boundaries and definitions between source and consuming projects, and discover how to avoid the "Kafka spiderweb" of duplicate datasets and their associated costs. Dive into the benefits of uniform datasets and learn how to implement them using Enterprise Domain Driven Design combined with Event Storming. Follow along with a demonstration illustrating this approach, and gain insights into topics such as CQRS for operational systems, DDD merged with event storming, eventing diagrams, and the differences between operational and data/analytics ecosystems. Understand the shift from monoliths to data mesh architectures and how this impacts real-time, ubiquitous, distributed data implementations.

Intro
Agenda
The problem
History of data segregation within apps
CQRS for operational systems
DDD for a single application – Generic example
DDD merged with event storming
Eventing diagram
Enterprise domain driven design
Operational vs Data/Analytics ecosystems
Monoliths vs Data mesh
Summary