Read as Needed - Building WiSER, a Flash-Optimized Search Engine

Explore the development of WiSER, a flash-optimized search engine, in this 31-minute conference talk from FAST '20. Discover how the "read as needed" philosophy drives the design of this innovative system, which leverages high-performance SSDs to deliver improved throughput and reduced latency. Learn about key techniques such as optimized data layout, a novel two-way cost-aware Bloom filter, adaptive prefetching, and space-time trade-offs. Understand how WiSER outperforms state-of-the-art alternatives like Elasticsearch, achieving up to 2.7x increased query throughput and 16x reduced latency, even with limited main memory. Gain insights into the potential applications of the "read as needed" approach as storage device read performance continues to improve.

Introduction
Search engines are overloads
Can we use more RAM
Read as Needed
Table
Contributions
Outline
Background
Phrase Mesh
Filtering
Adaptive Prefetch
Changes in Space
Evaluation
Questions
Bloom Filters
Search Queries