When the Magic Wears Off - Flaws in Machine Learning for Security Evaluations
Offered By: USENIX Enigma Conference via YouTube
Course Description
Overview
Explore a critical analysis of machine learning-based malware classification in this conference talk from USENIX Enigma 2019. Delve into the endemic issue of inflated results caused by spatial and temporal biases in experimental design. Learn about proposed space and time constraints for more accurate experiment design, and discover a new metric for evaluating classifier performance over time. Examine TESSERACT, an open-source evaluation framework that enables fair comparison of malware classifiers in realistic settings. Gain insights from a case study involving two well-known malware classifiers tested on a dataset of 129,000 applications, revealing result distortions due to experimental bias and demonstrating the benefits of performance tuning.
Syllabus
USENIX Enigma 2019 - When the Magic Wears Off: Flaws in ML for Security Evaluations
Taught by
USENIX Enigma Conference
Related Courses
The Data Scientist’s ToolboxJohns Hopkins University via Coursera MARS2014-1x: Metabolic Applied Research Strategy
Ethicon via Independent Experimentation for Improvement
McMaster University via Coursera Molecular Biology - Part 1: DNA Replication and Repair
Massachusetts Institute of Technology via edX Introduction to Linear Models and Matrix Algebra
Harvard University via edX