Using kNN for Predictive Analytics

kNN is non-parametric and instance-based and is used in supervised learning. It is a robust and versatile classifier, frequently used as a benchmark for complex classifiers such as Neural Networks (NNs) and Support Vector Machines (SVMs). kNN is commonly used in economic forecasting, data compression, and genetics based on their expression profiling.

Working Principles of kNN

The idea of kNN is that from a set of features x we try to predict the labels y. Thus, kNN falls in a supervised learning family of algorithms. Informally, this means that we are given a labeled dataset consisting of training observations (x, y). Now, the task is to model the relationship between x and y so that the function f: X→Y learns from the unseen observation x. The function f(x) can confidently predict the corresponding label y prediction on a point z by looking at a set of nearest neighbors.

However, the actual method of prediction depends on whether or not we are doing regression...