7.3 The singular value decomposition

So, we can diagonalize any real symmetric matrix with an orthogonal transformation. That’s great, but what if our matrix is not symmetric? After all, this is a rather special case.

How can we do the same for a general matrix? We’ll use a very strong tool, straight from the mathematician’s toolkit: wishful thinking. We pretend to have the solution, then reverse engineer it. To be specific, let A ∈ℝ^n×m be any real matrix. (It might not be square.) Since A is not symmetric, we have to relax our wishes for factoring it into the form UΛU^T . The most straightforward way is to assume that the orthogonal matrices to the left and to the right are not each other’s transposes.

Thus, we are looking for orthogonal matrices U ∈ℝ^n×n and V ∈ℝ^m×m such that

holds for some diagonal Σ ∈ℝ^n×m. (A non-square matrix Σ = (σ_i,j)_i,j=1^n,m is diagonal...