High Performance Data Processing with Python, Kafka and Elasticsearch

Explore high-performance data processing techniques using Python, Kafka, and Elasticsearch in this 24-minute EuroPython Conference talk. Dive into the Producer-Consumer model and its implementation for real-time data updates across platforms. Learn how to solve challenges in healthcare and e-commerce applications that require frequent data refreshes. Discover the basics of the Producer-Consumer pattern, its applications, and a deep-dive explanation using Elasticsearch and Kafka. Gain insights into creating an end-to-end framework in Python, including code walkthroughs for producing, streaming, and consuming data. Suitable for beginners and intermediate developers, this talk covers problem statements, system design, performance considerations, and concludes with a Q&A session to address your specific queries.

Introduction
Overview
Problem Statement
Drawbacks
Producer Consumer Model
App Flow
Data Store
Kafka response
Data update code
Final update
Conclusion