Cache à la Carte - A Framework for In-Memory Caching

Explore a comprehensive framework for in-memory caching in this 49-minute conference talk from Strange Loop. Dive into Twitter's approach to solving diverse caching challenges, including storing tiny objects, fast updates of large values, and replacing in-process cache in Java. Learn about the modular cache framework built with composable pieces, offering predictable tail latencies, fixed memory footprint, and readable logs and metrics. Discover how to implement new cache backends with minimal effort, including support for memcached and redis protocols. Gain insights from years of operating large-scale cache clusters in production, focusing on building operations-friendly systems. Understand the architecture, core decisions, and development process behind this framework, including code reuse, metrics management, and structural symmetry. Explore case studies on implementing SlimCache with cuckoo hashing, achieving significant metadata overhead reduction. Benefit from lessons learned in designing and implementing cache systems at scale, applicable to both Twitter-specific and general use cases.

Intro
ABOUT ME
ABOUT THE TALK
DEPLOYMENT & SCALE
A "SOLVED" PROBLEM
COMMON CHALLENGES
THE CACHE WE WANT
A DIRTY LITTLE SECRET
PREDICTABLE
OBSERVABLE
FLEXIBLE
ARCHITECTURE OVERVIEW
AND FOR EACH MODULE
JUDICIOUS CODE REUSE
CORE DECISIONS
DEVELOPMENT
LOG STATS. CONFIG
DECLARE/INITIALIZE METRICS
UPDATE METRICS
INCLUDE METRICS
BUFFER, CHANNEL, STREAM
STRUCTURAL SYMMETRY
STANDARD FLOW
CASE STUDY: HOW MUCH CODE
CASE STUDY:ADD SLIMCACHE
SLIMCACHE: CUCKOO HASHING
IN SUMMARY
FOR TWITTER
FOR THE REST
LESSONS LEARNED