Calculating the Value of Pie - Real-Time Survey Analysis With Apache Kafka

Explore real-time survey analysis using Apache Kafka® in this NDC Oslo conference talk. Learn how to create an interactive Telegram bot for conducting surveys, leveraging Kafka Producers and Consumers. Discover the power of ksqlDB for processing survey responses, including both stateless and stateful operations. Gain insights into data thinking, schema usage, and configuring Kafka components. Follow along as the speaker demonstrates a complete pipeline for calculating pastry preferences, showcasing practical applications of stream processing in everyday scenarios. Delve into Kafka architecture, Schema Registry, and Kafka Connect while understanding how to handle late results and create histograms. Perfect for developers interested in building efficient, real-time data analysis systems using Apache Kafka®.

Introduction
What are you baking
The problem statement
Surveys are everywhere
Survey analysis in general
Making it more exciting
What is Apache Kafka
Event Platform
Pie Example
Kafka Architecture
Kafka Cluster Example
Where to Start
Understanding the Data
Kafka Schema Registry
Kafka Survey IDs
Write Survey to Kafka
Conversation Handler
Message Handler
Send
Processing Data with Kafka
Define Inputs
Normalize Data
Survey Responses
Create Table
Create Histogram
Output
RealTime Results
Late Results
Kafka Connect
Summary
Baking something
Stream processing use cases