Deep Learning for Search and Recommender Systems in Practice

Explore deep learning techniques for search and recommender systems in this conference talk from KDD 2020. Dive into document retrieval methods, scoring and ranking algorithms, and personalization strategies. Learn about evaluation metrics like NDCG and MAP, and understand different approaches to learning-to-rank, including pointwise, pairwise, and listwise ranking. Discover the DeText framework for deep learning-based ranking and gain insights from LinkedIn experts on implementing these techniques in practice.

System Overview Document Retrieval Scoring and Ranking . Personalization and Re-ranking
Document Retrieval • Simple regex based retrieval . Traditional inverted index based retrieval Embedding based retrieval
Metrics for Evaluation • Multiple level of relevance NDCG (Normalized Discounted Cumulative Gain) . Binary relevance DMAP (Mean Average Precision) MRR Meon Reciprocal Ronk
Normalized Discounted Cumulative Gain Discounted Cumulative Goin
Mean Average Precision Precision: Relevant documents up to rank K/K
Mean Reciprocal Rank Reciprocal Rank
Learning to Rank
Pointwise Ranking Loss function is based on a single (query, document) pair
Regression based pointwise ranking Input (4.x) feature vector responding to the query and a document, Label: y relevance of the document
Classification based pointwise ranking
Ordinal regression based pointwise ranking
Summary of pointwise ranking Pros • Simple, considering one document at a time. • Available algorithms are rich. Most regression/classification algorithms can be used.
Pairwise Ranking Loss function is based on query and a pair of documents.
Listwise Ranking Loss function is based on the query and a list of documents
AdaRank Motivation: commonly used evaluation metrics are not differentiable. So it is not easy to optimize directly. AdaRank minimizes the exponential loss. El below can be NDCG.
List Net / ListMLE Map list of scores to a probability distribution by Plockett-Luce model. • Permutation probability, where 5() is the scoring function.
Summary of listwise ranking Pros
DeText: a Deep Learning Ranking Framework