RocksDB Advanced Features and Optimizations - Lightning Talks

Explore a series of lightning talks from the RocksDB Meetup in December 2016, covering diverse topics in database management and optimization. Learn about Lua Compaction Filters, Read Cache implementation, improved statistics and delete range functionality, iterator-assisted inserts, large value support, new iterator features, bulk loading techniques, high-speed RocksDB cloning with WDT, and the zero-copy iterator library Iterlib. Gain insights into advanced database concepts, performance optimizations, and practical applications of RocksDB in various scenarios, presented by experts from Meta.

Intro
Recap: Compaction A RocksDB operation that merges SST files
Recap: Compaction Filter A customizable logic to delete key-value pairs during compaction
Lua Compaction Filter You can write Compaction Filters in Lua!
How to Use?
CPU Overhead CPU usage in one Facebook internal use case
Limitations
Time-series data
Writing a key
Insert into memtable
Reduce CPU usage for binary search
Benchmarks
Key Benefit
Usage Open the DB
Design Considerations
Benchmark - (WA)
Future work.
What is SeekForPrev?
Motivation
Basic Logic What you do previously to achieve same functionality
Implementation sketch
Direct 1/0
Write Path - Memtable
Write Path - Flush
Write Path - Compaction
Bulkload-Create SST file
Bulkloading - Level criteria
Bulkload - memtable conflict
Use cases
Shard Migration DB1
Import from Hadoop
Bulkload - Create SST file
Bulkload - Ingest SST file
Cloning a DB
Before WDT...
Facebook internal usage of WDT (cont.) Bytes per minute per use case
Java 8 Streams
Secret Sauce?
Iterlib::dynamic
Iterators