Support Vector Machines

A Support Vector Machine (SVM) is a classifier that is defined using a separating hyperplane between the classes. This hyperplane is the N-dimensional version of a line. Given labeled training data and a binary classification problem, the SVM finds the optimal hyperplane that separates the training data into two classes. This can easily be extended to the problem with N classes.

Let's consider a two-dimensional case with two classes of points. Given that it's 2D, we only must deal with points and lines on a 2D plane. This is easier to visualize than vectors and hyperplanes in a high-dimensional space. Of course, this is a simplified version of the SVM problem, but it is important to understand it and visualize it before we can apply it to high-dimensional data.

Consider the following figure:

Figure 9: Separating two classes with a hyperplane

There are two classes of points and we want to find the optimal hyperplane to separate...