Statistical and Probabilistic Foundations of AI

The 'Statistical and Probabilistic Foundations of AI' course provides an accessible overview of the mathematics and statistics behind fundamental concepts of machine learning, data science, and artificial intelligence.

It covers descriptive and exploratory data analysis and a brief introduction to inferential statistics. Starting with summary statistics, it focuses on visualising data and the resulting key characteristics. This includes box plots, histograms, kernel density estimates, and regression. In addition, the course provides the principles of probability necessary to understand the methods used in inferential statistics and machine learning at an introductory level. Starting with the basic concepts of probability and elementary stochastic models, the course also covers more advanced topics of probability theory. These include multivariate distributions, generating functions, limit theorems, and a brief introduction to stochastic simulation.

Finally, a brief introduction to inferential statistics is given. Parametric and non-parametric inferential approaches are discussed. Point and interval estimation and hypothesis testing are also covered.

The presentation is rounded off with many examples and data that are analysed and visualised using R.

Week 1: Elementary Statistics

We will begin by introducing basic statistical vocabulary, which we will then use to provide methods for summarising and displaying data. We will use the process of data binning to provide histograms as well as providing standard key characteristics for location and deviation measures (e.g., means, quantiles, empirical variance).

Week 2: Elementary Statistics

In the second week we will continue with methods suitable for comparing and visualising data. We will introduce box plots and linear transformations as simple tools before exploring possible relationships between variables. Finally, we will look at regression models and related methods, focusing on (multiple) linear regression.

Week 3: Fundamentals of Probability

In the third week we will introduce the basic principles of probability. This includes the notion of sample spaces and the calculation of probabilities. We will also study the notion of conditional probability and the independence of random events.

Week 4: Random Variables

We will introduce the basic idea of random variables and their distribution as a way of modelling random events. We will also consider univariate discrete probability distributions before moving on to univariate probability distributions with Riemann density functions and multivariate distributions.

Week 5: Random Variables

Starting with the concept of multivariate distributions, we will consider their marginal and conditional distributions. We then move on to the concept of expectation of random variables and random vectors (for both discrete and continuous distributions). In particular, we study the properties of expected values as well as those of variance and covariance, respectively.

Week 6: Random Variables

The sixth week marks the end of the probability part, where we look at sums of random variables and ways of gaining insight into their distribution. In particular, we consider both the convolution and the generation function techniques. We finish with limit results for sequences of random variables, covering the strong law of large numbers and the central limit theorem.

Week 7: Inferential Statistics

In the final week we will introduce the core concepts of inferential statistics. Using probability theory, we will be able to construct point estimators and confidence intervals in a parametric statistical model. Some non-parametric examples will also be given. We will conclude the course with a brief introduction to statistical tests and selected statistical tests for Gaussian distributions.