Linear and Integer Programming

Linear Programming (LP) is arguably one of the most important optimization problems in applied mathematics and engineering. The Simplex algorithm to solve linear programs is widely regarded as one among the "top ten" algorithms of the 20th century. Linear Programs arise in almost all fields of engineering including operations research, statistics, machine learning, control system design, scheduling, formal verification and computer vision. It forms the basis for numerous approaches to solving hard combinatorial optimization problems through randomization and approximation.

The primary goals of this course will be to:

1. Understand the basic theory behind LP, algorithms to solve LPs, and the basics of (mixed) integer programs (ILP).

2. Understand important and emerging applications of LP and ILPs to economic problems (optimal resource allocation, scheduling problems), machine learning (SVM), and combinatorial optimization problems.

At the end of the course, the successful student will be able to cast various problems that may arise in her research as optimization problems, understand the cases where the optimization problem will be linear, choose appropriate solution methods and interpret results appropriately. This is generally considered a useful ability in many research areas.

Introductory Material 

• Introduction to Linear Programming.

• The Diet Problem.
• Linear Programming Formulations.
  
• Tutorials on using GLPK (AMPL), Matlab, CVX and Microsft Excel.
• The Simplex Algorithm (basics).

• Handling unbounded problems
• Degeneracy
• Geometry of Simplex
• Initializing Simplex.
• Cycling and the Use of Bland's rule.

• Duality: dual variables and dual linear program.
• Strong duality theorem.
• Complementary Slackness. 
• KKT conditions for Linear Programs.
• Understanding the dual problem: shadow costs.
• Extra: The revised simplex method.

• Advanced LP formulations: norm optimization.
• Least squares, and quadratic programming.
• Applications #1: Signal reconstruction and De-noising.
• Applications #2: Regression.

• Integer Linear Programming.
• Integer vs. Real-valued variables.
• NP-completeness: basic introduction.
• Reductions from Combinatorial Problems (SAT, TSP and Vertex Cover).
• Approximation Algorithms: Introduction.

• Branch and Bound Method
• Cutting Plane Method

• Applications: solving puzzles (Sudoku), reasoning about systems and other applications.
• Classification and Machine Learning