How Can You Trust Machine Learning?

Explore a comprehensive framework for building trust in machine learning and AI systems in this Stanford University seminar. Delve into Professor Carlos Ernesto Guestrin's discussion on the three pillars of clarity, competence, and alignment that can lead to more effective and trustworthy AI. Examine real-world examples, including visual question answering, type 1 diabetes management, and image classification, to understand the challenges and solutions in creating interpretable and reliable ML models. Learn about techniques such as explanations for neural network predictions, data augmentation, and adaptive loss alignment. Discover how to address the increasing complexity of foundation models and optimize for multiple metrics to enhance the trustworthiness of AI systems in various applications.

Intro
Math Myth of ML (circa 2008)
spaces between the Math
trust for whom?
Train a neural network to predict wolf v. husky
Explanations for neural network prediction
Accuracy vs Interpretability
Explaining predictions
Explaining prediction of Inception Neural Network
Anchors for Visual Question Answering
Type 1 Diabetes Management
Standard Intervention
Oversensitivity in image classification
Beyond Test-Set Accuracy
Closing the Loop with Simple Data Augmentation
Checklist: Test Linguistic Capabilities of Model
Checklist: Categories of Tests
Addressing Challenge of Test Creation
User Study: Quora Question Pairs (n=18, 2 hours)
Minding the Gap
Adaptive Loss Alignment (ALA)
And this gap is increasing with foundation models...
Optimizing for multiple metrics