Tackling Covariate Shift with Node-Based Bayesian Neural Networks

Explore a 29-minute conference talk by Trung Trinh on tackling covariate shift using node-based Bayesian neural networks. Delve into the challenges of weight-based BNNs and discover how node-based alternatives offer scalable solutions for improved generalization under covariate shift. Learn about the interpretation of latent noise variables as implicit representations of data perturbations and their impact on BNN performance. Examine the relationship between latent variable entropy and implicit corruption diversity, and understand a proposed approach to increase this entropy during training. Gain insights into out-of-distribution image classification benchmarks and improved uncertainty estimation under covariate shift due to input perturbations. Explore topics such as covariate shift, Bayesian neural networks, node-based BNNs, implicit corruptions, variational inference, and robust learning under label noise. Benefit from the expertise of Trung Trinh, a Ph.D. student in the Probabilistic Machine Learning group at Aalto University, as he shares his research on using Bayesian methods to quantify predictive uncertainty in deep learning models.

Intro
Covariate shift
Bayesian neural networks (BNNs)
BNNS perform worse than MAP models under corruption
Node-based Bayesian neural networks
Approximating the implicit corruption
Example of implicit corruptions
Entropy of latent variables and implicit corruptions
Is a model robust against its own corruptions?
How robust is a model against the other model's corruptio
Training a node-based BNN
Variational inference
Variational posterior
Training objective
Effects of on corruption robustness
Robust learning under label noise
Benchmark comparison
Conclusion