Machine Learning for Relevance and Serendipity

Explore the intricacies of machine learning systems for personalized content discovery in this 39-minute conference talk from Strange Loop 2013. Delve into Prismatic's real-time personalization engine, designed to provide users with both expected content and targeted serendipity. Learn about sophisticated techniques for topical classification, story similarity, topic suggestions, and social connection value assessment. Gain insights into the personalized relevance model, including problem formulation, feature design, model design, evaluation metrics, and experimental setup. Discover how the combination of social cues, topical classification, publisher information, and user interaction analysis produces highly relevant and serendipitous content. Understand the challenges in machine learning development, including data bugs, presentation bias, and statistical phenomena like Simpson's Paradox. Presented by Jenny Finkel, Chief Software Architect at Prismatic, this talk offers valuable knowledge for those interested in leveraging machine learning for enhanced user experiences in content discovery platforms.

Machine Learning for Relevance and
The Discovery Problem
How we do it
People who use the term 'secret sauce' know about neither
Talk Outline
Our Topic Classifier
Interest Scores
Saved Session Data
You are all unique snowflakes
How do we know if we've done a good job?
Formula for a Machine Learning Problem
Collect Data • Save session information (described earlier)
Featurize Data
Training the model (numeric optimization)
Classify New Data
Original formulation weight vector is universal, features are user-specific
The Development Cycle
Data bugs (are the worst)
Presentation bias
Ranking vs Normal ML
Statistical Bleeding
Simpsons Paradox
Summary • We collect information about our users' preferences and dispreferences
Prismatic Backend Team