The Role of Data and Models for Deep-Learning Based Image Reconstruction

Explore the role of data and models in deep learning-based image reconstruction through this insightful lecture by Reinhard Heckel from the Technical University of Munich. Delve into the impact of model size and training data on performance, particularly in accelerated magnetic resonance imaging. Examine the robustness of deep learning methods compared to classical reconstruction techniques, and investigate their performance under distribution shifts. Discover strategies to improve out-of-distribution performance, including the use of diverse training data and test-time-training. Gain valuable insights into scaling challenges, dataset sizes in imaging tasks, and the behavior of various reconstruction methods such as U-net-based denoising and Swin transformer-based denoising.

Intro
Scaling language models
Data set sizes in imaging tasks are small
Expected performance behavior
U-net-based denoising
U-net-based accelerated MRI
Swin transformer based denoising
Reconstruction methods
What we might expect
Dataset shift
Adversarially filtered shift
Goal: improve performance unter distribution shifts
For classification problems, "natural distribution shifts are an open research problem"
Improving performance for 11-minimization is easy
Test time training
Closing the distribution shift performance gap for anatomy shift
Closing the distribution shift performance gap with test-time-training
References