Building a Measurement System for Personalization - A Bayesian Approach

Learn how to build an effective measurement system for personalization programs using a Bayesian approach in this 33-minute conference talk from the Toronto Machine Learning Series. Explore the challenges of measuring personalization effectiveness and discover how Sobeys developed an experimentation and measurement platform to evaluate treatment variants across customer segments. Gain insights into configurable, automated processes for repeated experimentation, and understand why Bayesian statistics offer advantages over frequentist approaches for hierarchical experiments. Delve into topics such as customer assignment optimization, basics of Bayesian statistics, attribution of lift to program components, and the adoption of MLOps-like approaches in measurement platform design. Benefit from practical examples and learn how to address issues like Simpson's Paradox in complex hierarchical data analysis.

Intro
About Personalization at Sobeys
Our Measurement Objectives
The Experiment Set-Up
Comparing Approaches to Inference
The Bayesian Approach to Inference
Why Bayesian Inference?
Introducing the Example!
A Simple Model
Adding the Engagement Group
Partial Pooling: Effect Results
Closing Remarks