2.2 Inner products, angles, and lots of reasons to care about them

In the previous section, we imbued our vector spaces with norms, measuring the magnitude of vectors and the distance between points. In machine learning, these concepts can be used, for instance, to identify clusters in unlabeled datasets. However, without context, distance is often not enough. Following our geometric intuition, we can aspire to measure the similarity of data points. This is done by the inner product (also known as the dot product).

You can recall the inner product as a quantity that we used to measure the angle between two vectors in high school geometry classes. Given two vectors x = (x₁,x₂) and y = (y₁,y₂) from the plane, we defined their inner product by

for which it can be shown that

⟨x,y⟩ = ∥x∥∥y∥ cosα (2.1)

holds, where α is the angle between x and y. (In fact, there are two such angles, but their cosine is equal....