Statistical Physics and Statistical Inference - Lecture 2

Explore statistical physics and statistical inference in this comprehensive lecture by Marc Mézard, Director of Ecole normale supérieure - PSL University. Delve into Bayesian inference, error-correcting codes, and phase transitions in spin glasses. Examine crystal hunting algorithms, belief propagation, and mean-field equations. Investigate applications in compressed sensing, perceptron learning, and generalized linear regression. Learn about message passing algorithms, TAP equations, and strategies for overcoming glass traps in optimization problems. Gain insights into the connections between statistical physics and machine learning through this in-depth presentation, which includes a Q&A session.

DATE: 06 January 2020, 16:00 to
Lecture 1 Public Lecture: 6 January 2020, PM
Lecture 2: Tuesday 7th January 2020, PM
Lecture 3: Wednesday 8th January 2020, PM
Statistical physics and statistical inference Lecture 2
What is inference?
Bayesian inference
Efficient codes : parity checks LDPC codes
Error decoding: crystal hunting inference problem
Error decoding: inference problem
Phase Transitions in Error correcting codes
Error correction: decoding
Phase transitions in decoding
Statistical inference: general scheme
Bayesian inference with many unknown and many measurements
Spin glasses
Phase transition with many states: spin glasses
Inference, spin- glass and crystal: tomography of binary mixtures
Tomography of binary mixtures
Crystal : much more probable
Inference with many unknowns crystal hunting with mean-field based algorithms
Historical development of mean field equations
BP = Bethe-Peierls = Belief Propagation
BP equations
Infinite range models :
Example: SK model
SK model, TAP equations
Two important developments
2 What happens in a glass phase, when there are many pure states, and therefore many solutions?
SP=BP 2
Power of message passing algorithms
An example of fully connected model: Generalized Linear Regression
Perceptron learning
Compressed sensing
Spin glass with multispin interactions, infinite range
BP equations
TAP equations
Benchmark: noiseless limit of compressed sensing with iid measurments
Analysis of random instances : phase transitions
Design the matrix so that one nucleates the naive state crystal nucleation idea,
Getting around the glass trap
Nucleation and seeding
Structured measurement matrix.Variances of the matrix elements
Numerical study
Performance of AMP with Gauss-Bernoulli prior: phase diagram
Many glass states
Q&A