Managing Reference Data with Sirius - Not-So-Big Data After All

Explore the concept of managing reference data with Sirius in this Philly ETE 2014 conference talk. Learn about the definition of reference data and its applications, understand how Moore's Law impacts data management, and discover the architecture of Sirius-based applications. Dive into a case study from Comcast showcasing Sirius's capabilities in serving millions of customers. Gain insights into accessing and publishing reference data, operational flexibility, and cluster management. Examine the Scala and request handler interfaces, update and read paths, and key-value operations. Understand compacting strategies, native data structures, and persistence replay. Discover the open-source nature of Sirius and explore topics such as serialization, bootstrapping reference data, and update ordering.

Intro
Definition of Reference Data
Example of Reference Data
Moores Law
Encyclopedia Britannica
Wikipedia
Accessing Reference Data
The Model
Publishing Reference Data
Recovery
Operational Flexibility
Dependent Clusters
Scala Interface
Request Handler Interface
Update Path
Read Path
Key Value Interface
Compacting During Deployment
Compacting in the Background
Pseudocode
Wrap Up
X2 Interface
Native Data Structures
Persistence Replay
Open Source
Mirror vs Cache
Serialization
Cluster Management
Questions
Bootstrapping Reference Data
How many keys do you need
Ordering of updates