Better Decisions with Machine Learning - Peter Flach

Explore the intersection of machine learning and decision-making in this comprehensive lecture from the Alan Turing Institute's conference on decision support and recommender systems. Delve into topics such as majority class decision rules, ROC curves, logistic and beta calibration, precision-recall-gain curves, and F-score calibration. Learn how to adapt machine learning models to deployment contexts and gain insights into the development of a measurement theory for ML. Discover the potential of AI techniques in supporting complex decision-making processes across various domains, including management, health, urban planning, and sustainability.

Intro
Decisions, decisions..
Outline of the talk
Majority class decision rule
Adapting to deployment context 17-1/31
1. Introducing ROC curves and calibration
Logistic calibration from first principles
Example of inverse-sigmoidal distortion
Beta calibration from first principles
A rich parametric family
Beta-calibration is easily implemented
Precision-Recall-Gain Curves
Model calibrated for F-score In-1/21
ROC curves and Precision-Recall curves
Properties of ROC curves
F-score calibration
Perspective: Towards a measurement theory for ML
Measuring things
Concatenation and scales
Concatenating confusion matrices