Beyond Testset Performance - Strategies for Clinical Deployment

Explore strategies for clinical deployment of AI models in medicine beyond testset performance in this insightful conference talk. Delve into the challenges of translating academic research success to real-world clinical practice, examining various studies and papers that highlight risks and obstacles. Learn about developing robust evaluation frameworks and monitoring systems to enhance model reliability for deployment. Gain valuable insights on label-efficient models, observational supervision, self-supervision, and methods to assess and improve model trust and robustness to distribution shifts. Engage in a critical discussion on key topics in AI and medicine, generating fresh ideas for their intersection. Benefit from the speaker's expertise in machine learning methodology for medical applications and her research focus on building reliable models for clinical use.

Introduction
Overview
Model Performance
Objectives
Clever Hands Effect
Beyond Testset Performance
Failure Modes
Current Day Models
Clinical Deployment Papers
Use Cases
Integration with Human Experts
Algorithm Aversion
Radiology
Paper
Framings
Variables
Survival Analysis
Takeaways
Strategies for Clinical Deployment
Benefits of Better Testsets
Other Strategies
Incremental Learning
Federated Learning
Designing Human AI Collaboration
Personal comments
What are doctors looking for
Thoughts on AIbased biomarkers
Wisdom from the crowd