Right at the outset of this chapter, we defined the modus operandi of machine learning algorithms. If you recall, we had said that an ML system is presented with training data. It then makes its own set of rules or builds a model, which it uses to further make predictions on unseen (test) data. By revisiting this definition, I want to focus on the two key things that an ML algorithm can do with the training data:
Formulate a set of rules.
Build a model.
We have covered the basics of the k-nearest neighbor classifier in great detail. Let's try to place the operation of the kNN algorithm in the context of the two points we have listed above. Given the training data and a query point to classify, the kNN looks at the neighboring points and decides the class of the query point based on a majority vote. Clearly, this is a case of an ML algorithm that applies a set of rules based on the training data it has been presented with for the purpose of classifying...
