Hyper Speed - When Big Data Blooms

Explore the challenges of processing massive data volumes in this NDC Oslo 2021 conference talk. Dive into Report URI's infrastructure and learn how Bloom Filters, an often-overlooked data structure, enable handling immense amounts of data with exceptional speed and space efficiency. Discover techniques for processing up to 65,000 JSON payloads per second containing global client telemetry. Gain insights into cross-site scripting, Coinhive, Magecart, Content Security Policy, and CSP reports. Understand the implementation of Bloom Filters, including insertion methods, probabilities, and width considerations. Examine real-world applications through case studies from Akamai and Google Safe Browsing. Explore additional data structures like Count-Min Sketch, Top-K, and HyperLogLog for big data analysis.

Intro
Title
The Problem
Crosssite Scripting
Coinhive
Magecart
Content Security Policy
JavaScript
Content Security
Report Uri
CSP Reports
How can we do this
Bloom Filters
Inserting Items
Probabilities
Bloom Filter Width
Bloom Filters Online
Akamai Case Study
Google Safe Browsing
Medium
Counting
Sketch
TopK
Hyperlog Log
QA Session