YoVDO

When the Magic Wears Off - Flaws in Machine Learning for Security Evaluations

Offered By: USENIX Enigma Conference via YouTube

Tags

Machine Learning Courses Experimental Design Courses Malware Classification Courses

Course Description

Overview

Explore a critical analysis of machine learning-based malware classification in this conference talk from USENIX Enigma 2019. Delve into the endemic issue of inflated results caused by spatial and temporal biases in experimental design. Learn about proposed space and time constraints for more accurate experiment design, and discover a new metric for evaluating classifier performance over time. Examine TESSERACT, an open-source evaluation framework that enables fair comparison of malware classifiers in realistic settings. Gain insights from a case study involving two well-known malware classifiers tested on a dataset of 129,000 applications, revealing result distortions due to experimental bias and demonstrating the benefits of performance tuning.

Syllabus

USENIX Enigma 2019 - When the Magic Wears Off: Flaws in ML for Security Evaluations


Taught by

USENIX Enigma Conference

Related Courses

The Data Scientist’s Toolbox
Johns Hopkins University via Coursera
MARS2014-1x: Metabolic Applied Research Strategy
Ethicon via Independent
Experimentation for Improvement
McMaster University via Coursera
Molecular Biology - Part 1: DNA Replication and Repair
Massachusetts Institute of Technology via edX
Introduction to Linear Models and Matrix Algebra
Harvard University via edX