Dynamic Meta-Learning for Anomaly Detection - Cole Sodja, Microsoft Defender ATP

Explore a methodology for measuring probabilistic calibration and dynamically updating scores in anomaly detection systems. Delve into the challenges faced by Microsoft Defender Advanced Threat Protection in monitoring billions of anomaly detectors across diverse data sources. Learn about adaptive mixtures of functional inflated beta-binomial models and their application in identifying and updating scores for cyber indicators of attack. Discover how an automated statistical diagnostics system can infer optimal weighting of anomaly scores through probabilistic inference, addressing the scalability issues of manual detector review. Cover topics such as p-value calibration, model uncertainty, meta-learning, Bayesian approaches, state-space models, filtering, and probability distribution modeling in this comprehensive 1-hour 26-minute talk by Cole Sodja from Microsoft Defender ATP, presented at the Alan Turing Institute.

Introduction
Agenda
Motivation
Calibration
Pvalue calibration
Model uncertainty
What is calibration
MetaLearning
Bayesian Approach
Monitoring
Statespace Models
Filtering
Modeling
Probability distribution
Wrap up