7.7 Summary

I told you that climbing the peak is not easy: so far, this was our hardest chapter yet. However, the tools we’ve learned are at the pinnacle of linear algebra. We started by studying two special transformations: the self-adjoint and orthogonal ones. The former ones gave the spectral decomposition theorem, while the latter ones gave the singular value decomposition.

Undoubtedly, the SVD is one of the most important results in linear algebra, stating that every rectangular matrix A can be written in the form

where U ∈ℝ^n×n, Σ ∈ℝ^n×m, and V ∈ℝ^m×m are rather special: Σ is diagonal, while U and V are orthogonal.

When viewing A as a linear transformation, the singular value decomposition tells us that it can be written as the composition of two distance-preserving transformations (the orthogonal ones) and a simple scaling. That’s quite a characterization!

Speaking of singular values and eigenvalues...