DART - A Scalable and Adaptive Edge Stream Processing Engine

Explore DART, a scalable and adaptive edge stream processing engine, in this conference talk from USENIX ATC '21. Learn about the challenges faced by traditional data processing systems in handling time-critical and dynamically changing IoT applications. Discover how DART introduces a dynamic dataflow abstraction using distributed hash table (DHT) based peer-to-peer (P2P) overlay networks to automatically place, chain, and scale stream operators. Understand the engine's ability to reduce query latency, adapt to edge dynamics, and recover from failures. Examine DART's performance compared to Storm and EdgeWise, and its significant improvements in scalability, adaptability, and application deployment setup times for IoT applications on edge platforms.

Intro
What is Edge Stream Processing?
Why Edge Stream Processing?
Outline
Our Goal and Challenges
Challenge #1: How to scale to #applications?
Challenge #2: How to adapt to edge dynamics?
DART Design and implementation
Dynamic Dataflow Abstraction
Elastic Scaling Mechanism
Failure Recovery Mechanism
Performance Evaluation
Conclusion