Statistical Learning Theory for Modern Machine Learning - John Shawe-Taylor

Explore statistical learning theory for modern machine learning in this comprehensive seminar by John Shawe-Taylor from University College London. Delve into the foundations of learning generalization, high-confidence error distribution, and mathematical formalization. Examine risk measures, PAC-Bayes framework, and its comparison to Bayesian learning. Investigate the General PAC Bayesian Theorem and its proof, followed by an analysis of linear classifiers and SVM bounds. Gain insights from deep network training experiments and their results. Conclude with a discussion on the flexible framework of statistical learning theory and its implications for contemporary machine learning approaches.

Intro
Learning is to be able to generalise
Statistical Learning Theory is about high confidence
Error distribution picture
Mathematical formalization
What to achieve from the sample?
Risk (aka error) measures
Before PAC Bayes
The PAC-Bayes framework
PAC Bayes aka Generalised Bayes
PAC Bayes bounds vs. Bayesian learning
A General PAC Bayesian Theorem
Proof of the general theorem
Linear classifiers
Form of the SVM bound
Slack variable conversion
Observations
Deep Network Training Experiments
Training and Generalisation Results
A flexible framework
Conclusions