Shrinkage-Delinkage Trade-off in Gaussian Approximations for Variational Inference - UAI 2023 Oral Session 5

Explore the intricacies of factorized approximations in variational inference through this 27-minute conference talk from the Uncertainty in Artificial Intelligence (UAI) 2023 Oral Session. Delve into the analysis of the shrinkage-delinkage trade-off in Gaussian approximations, examining how factorized approximations tend to underestimate uncertainty in distributions. Investigate two key measures of uncertainty deficit: underestimation of componentwise variance and entropy. Learn about the explicit analysis of approximating a Gaussian with a dense covariance matrix by a Gaussian with a diagonal covariance matrix. Discover the competing forces that determine the entropy of the approximation and how the trade-off manifests in various scenarios, including high-dimensional problems. Gain insights into the relationship between entropy gap, problem dimension, and correlation matrix condition number. Examine empirical results on both Gaussian and non-Gaussian targets to validate the analysis and explore its limitations.

UAI 2023 Oral Session 5: Shrinkage Delinkage Trade-off In Gaussian Approx. for Variational Inference