Exploring Dense Vector Search at Scale

Explore the potential and challenges of implementing dense vector search at scale in this 43-minute conference talk from Haystack US 2024. Delve into the transformative impact of vector search on mainstream search applications, including its ability to search for meaning rather than words and its potential to evolve traditional search engines into answer engines. Learn about the balancing act between cost, speed, and quality when adopting vector search, and discover insights on handling production-like query loads, stress testing, and integrating vector search functionality into existing search pipelines. Follow along as the speakers share a reusable method for testing index and query performance using a Lucene HNSW index, while methodically adjusting common vector search variables. Gain valuable knowledge for product teams new to vector search, helping to avoid surprises in production environments.

Haystack US 2024 - Tom Burgmans & Mohit Sidana: Exploring dense vector search at scale