The Measure and Mismeasure of Fairness

Explore the complexities of fairness in machine learning decisions through this 46-minute lecture by Sharad Goel from Stanford University. Delve into recent developments in fairness research, examining quantitative methods for measuring bias in algorithmic decision-making. Investigate a detailed case study on pretrial detention, analyzing key assumptions, risk distributions, and threshold rules. Examine popular mathematical definitions of fairness, including classification parity and false positive rate parity. Critically evaluate the challenges of applying these fairness metrics, such as infra-marginality and potential biases in data labels and predictors. Gain insights into the nuanced considerations necessary for developing and assessing fair machine learning systems in real-world applications.

Quantifying bias in machine decisions
Summary
Pretrial detention A detailed case study
Key assumptions
From features to decisions
Risk distributions
From risk to decisions Threshold rules
A double standard
Fairness of a single threshold
Popular mathematical definitions of fairness
Discrimination with calibrated scores
Classification parity
False positive rate parity
Error rate disparities in Broward County
Calculating false positive rates
Infra-marginality
The problem with false positive rates
Are the data biased?
Biased labels
Biased predictors