Neural Network Verification as Piecewise Linear Optimization

Explore neural network verification as a piecewise linear optimization problem in this 26-minute conference talk by Joseph Huchette from Rice University. Delve into the challenges of ensuring robustness against imperceptible attacks in deep learning models, particularly in critical applications. Learn about framing verification tasks using linear programming (LP) and mixed-integer programming (MIP) techniques. Discover a framework for creating strong LP and MIP formulations for neurons with convex piecewise linear activations, and its application to ReLU networks. Examine the verification of binarized neural networks and the development of cutting planes to enhance MIP solve time and reduce search tree size. Gain insights into key concepts such as optimization over trained neural networks, deep reinforcement learning applications, and important theoretical results in the field. This talk, part of the Deep Learning and Combinatorial Optimization 2021 series at the Institute for Pure & Applied Mathematics (IPAM), offers a comprehensive overview of current research in neural network verification and its intersection with optimization techniques.

Intro
Neural network verification
Key insights and approach
Optimization over a trained neural network
Fitting unknown functions to make predictions
Application: Deep reinforcement learning
Application: Designing DNA for protein binding
Neural networks in one slide
Most important theoretical result
MIP formulations for a single ReLU neuron
MIP formulation strength
Formulations for convex PWL functions
Network 1: Small network standard training
Propagation algorithms
Computational results
Extensions: Binarized and quantized networks