Apache Kafka and KSQL in Action - Let’s Build a Streaming Data Pipeline

Explore the power of Apache Kafka and KSQL in this 30-minute Devoxx conference talk. Learn how to build a streaming data pipeline without extensive programming knowledge. Discover the architectural benefits of Apache Kafka as a scalable, fault-tolerant streaming platform. Dive into real-time integration using Kafka Connect to seamlessly integrate with various data stores. Witness the simplicity of stream processing with KSQL's SQL-like interface. Follow along as the speaker demonstrates filtering events in real-time, enriching data streams with RDBMS information, and leveraging streaming aggregates for anomaly detection. Gain insights into topics such as Kafka APIs, applications, and stream processing. Observe the creation of Kafka topics, KSQL topics, and continuous queries. Understand the process of deserializing messages, viewing live data, and working with stream schemas. Learn about extended ratings, power ratings, and message transformation. Explore Kafka Connect features, including JSON connectors and database integration. Witness a live demo showcasing the entire process, from creating Kafka topics to modeling data and connecting to databases. Walk away with a comprehensive understanding of building efficient streaming data pipelines using Apache Kafka and KSQL.

Introduction
What is Apache Kafka
Moving from an older system to a new one
Writing new services in Kafka
Kafka API
Kafka Applications
Platform
Stream Processing
Create a Kafka Topic
KSQL Topics
Deserialize Messages
Viewing Live Messages
Streams
Stream Schema
Continuous Queries
Create Stream
Extended Ratings
Power Ratings
CaseEqual
Filter Messages
Aggregate Messages
transform
Kafka Connect
Pair JSON
Connectors
In Action
Demo
Kafka Topic
Kafka Model
Connecting to the Database
Model a Kafka Topic
Before and After
Half Go Summit
Outro