Online Experimentation and AB Testing

Explore online experimentation and A/B testing in this comprehensive one-hour meetup. Gain a solid foundation in statistical fundamentals essential for understanding A/B testing, and learn how it's applied in online businesses. Discover the differences between A/B testing and multivariate testing, and understand key concepts such as control groups, treatment groups, hypothesis testing, and confidence intervals. Delve into metric evaluation, segmentation, and common pitfalls in running online experiments. Led by Artem Kopelev, an analytics expert with experience in healthcare research, health insurance, and local government, this talk provides valuable insights for anyone interested in data-driven decision making in digital environments.

Intro
Agenda
Background in Clinical Trials
What is A/B Testing?
Obama 2012 Campaign
Testing Call-to-Action Button
Testing Navigation Bar
Jocelyn or Michael?
Away Find - Mobile notifications for priority messages
Online Form
CALIFORNIA CLOSET
Why We Use A/B Testing
Multi-Variate Testing
A/B Testing vs Multivariate Testing
Control and Treatment
Metrics Used For A/B Testing
Null vs Alternate Hypothesis
Type I and Type II Error
The Cook and Smoke Detector
The Boy Who Cried Wolf
Confidence Interval Intuition
Calculating Confidence Interval
Steps in Experimentation
Categories of Metrics
A/B Testing Tools
Questions?