Inside Twitter's Realtime Delivery of Tweets for Enterprise Customers

Explore the intricate architecture behind Twitter's PowerTrack, a product designed for Enterprise customers to filter and receive customized subsets of Tweets in real-time. Delve into the technical challenges and solutions for filtering data at scale, ensuring application and system layer redundancy, applying configuration changes to open streaming connections, backfilling data for 100% fidelity, and managing customer stream states across data centers. Learn from Lisa White, tech lead of Twitter's streaming data products team, as she shares insights on the complex process of delivering millions of daily Tweets to hundreds of customers with thousands of end-users without missing a single Tweet. Gain a comprehensive understanding of the PowerTrack system, including its public firehose, customer rule predicate indexing, rule evaluation, index architecture, messaging queue, filtering layer, rules table, and state management. This conference talk from Strange Loop offers a deep dive into the technical intricacies of large-scale, real-time data delivery systems.

Introduction
What is PowerTrack
PowerTrack Architecture
Public Firehose
Customer Rule
predicate indexing
rule evaluation
index
architecture
messaging queue
filtering layer
rules table
state management
QA