Kolibri: An Execution Framework and UI for E-commerce Search Evaluation

Explore Kolibri, an open-source execution framework and UI for e-commerce search offline evaluations, in this comprehensive talk. Learn about the systematic approach to offline evaluation and optimization as search solutions grow, addressing the effects of parameter changes, tuning impacts on result quality, and optimal configuration determination. Discover how Kolibri works with any JSON-responding target system, offering UI-supported no-code definition of requests, result granularity, and evaluation. Delve into standard IR metrics, custom measures, and the importance of customization. Understand Kolibri's design, practical use cases, and potential applications in indexing, load testing, and simulation. Gain insights into the motivation behind Kolibri, its evaluation scheme, requesting process, selectors, tags, processing, metrics, and cluster setup. Explore the search endpoint, extensions, and participate in a Q&A session to deepen your understanding of this versatile search toolbox.

Introduction
Kolibri
Where can you run it
General Stack
Features
Motivation
Evaluation scheme
Requesting
Selectors
Tags
Processing
RunValue
ByrunningValue
Metrics
Cluster setup
Search endpoint
Extensions
Questions
Overview