Matrix Calculus for Linear Algebra - MIT 18.06 Spring 2020

Explore matrix calculus in this comprehensive lecture from MIT's Linear Algebra course. Delve into scalar calculus, linearization, gradients, and geometric interpretations. Learn about matrix/vector product rules and sophisticated gradient calculation methods. Examine specific examples, including f(x) = (Ax-b)'(Ax-b), and understand gradient notation and the trace concept. Investigate linear functions of matrices, gradients of matrix-to-scalar functions, and vector-to-vector Jacobians. Discover practical applications of gradients and explore Jacobian matrices for vector-to-vector and matrix-to-matrix functions. Gain insights into the relationship between Jacobians and volumes, and learn why writing out matrix elements isn't always necessary.

Matrix Calculus.
Scalar Calculus.
Emphasis on Linearization.
Gradients.
Geometrically.
Matrix/Vector Product Rule.
Gradients the straightforward but klunky way.
Gradients the sophisticated way.
Example f(x) = (Ax-b)'(Ax-b).
Gradient Notation.
The Trace.
Linear Functions of Matrices.
Gradients of Functions from Matrices to Scalars.
Vector to Vector Jacobians.
How are Gradients Used.
The Jacobian Matrix, vectors to vectors.
A Key Point -- you don't have to write out the matrix elements.
relationship to volumes.
Matrices to Matrices.
Derivatives of Matrix to Matrix Functions.