Towards Generalist Imaging Using Multimodal Self-Supervised Learning - Mars Huang

Explore cutting-edge approaches to generalist medical imaging AI in this Stanford University lecture by Mars Huang. Delve into the challenges of developing effective medical imaging models without large-scale labeled datasets and discover innovative solutions combining multimodal fusion techniques with self-supervised learning. Learn about strategies for training generalist models applicable across various tasks, modalities, and outcomes in healthcare automation. Gain insights into the speaker's research on pulmonary embolism detection, including PENet fusion types, global and local representation learning, and zero-shot classification results. Understand the potential for generalizing these techniques to other imaging modalities and patient cohorts, paving the way for more efficient and versatile medical AI systems.

Intro
Increase of Medical Imaging Utilization Can Hurt Patient
Limitation 1: Supervised learning requires large sc labeled datasets
Limitation 2: Few Medical Imaging Models Consider Clinical Context
Prototyping Methods Using Cohort of Pulmonary Embolism Patients
Specific Aims
Challenges For Pulmonary Embolism Detection
PENet
Fusion Types
Major types of self-supervised method for images
Learning global representations can be limiting
Global & Local Representations for Images using Attention G
Representation Learning Objective
Retrieval Results
Fine-tune Classification
Strategies for Generating Class Prompts
Zero-shot Classification Results
Next Steps
Generalizability to Other Downstream Tasks
Demonstrate feasibility of applying the propose to other imaging modalities and patient cohort