What is it?

KNN is a non-parametric supervised learning algorithm, which operates on the principle of similarity. Specifically, KNN algorithm predicts the label or value of a new data point by examining the labels or values of its K nearest neighbors in the training dataset.

In the training phase, KNN stores the entire training dataset as a reference. When making predictions, it computes the distance between the input data point and all training examples, using a chosen distance metric.

Subsequently, the algorithm identifies the K nearest neighbors to the input data point based on their distances. In classification tasks, it assigns the most common class label among the K neighbors as the predicted label for the input data point. In regression tasks, it calculates the average or weighted average of the target values of the K neighbors to predict the value for the input data point.

Figure 1 illustrates the operational mechanism of the KNN algorithm on a classification task.