Scaling Validation and Quality of Streaming Data Products at Twitter

Explore the intricacies of scaling validation and quality assurance for streaming data products at Twitter in this 33-minute conference talk from Strange Loop. Dive into the challenges faced by the Data Products Quality Engineering team as they build and maintain services, tools, and processes to support feature teams in providing a robust distributed streaming architecture. Learn about the evolution from simple QA to sophisticated tools for identifying issues in pre-production and live microservices. Discover the team's specialized approaches, including end-to-end stress testing, measuring pipeline data loss against product SLAs, and per-message data quality assessment. Gain insights into the value these tools and processes bring to engineers and Twitter's data customers. Follow along as the speaker covers topics such as staging environments, operating constraints, testing methodologies, and the development of better testing tools to meet the growing demands of Twitter's data business.

Introduction
Good nip at Twitter
Quality engineering
Staging environment
Operating constraints
DDI DNS adage
Testing for data products
PowerTrack
Two problems to solve
Puli as hell to solve
Experimenting with embedding
Testing as a service
Five factor testing
Better testing tools
Overhead
What Were We Missing
Stress Testing
Load Testing
Load Tool
Reinventing Tools