Describe types of algorithms associated with ML, including:
K-nearest neighbour
K-Nearest Neighbour (KNN) is a supervised machine learning algorithm used for classification and sometimes regression.
It is based on the idea that “similar things are near each other”. In classification, KNN looks at the k nearest data points (neighbours) to a new sample and uses majority voting to assign a class label.
You have labelled data: features + known labels.
A new data point arrives. KNN calculates the distance between this new point and all existing data points.
It picks the k closest points (based on Euclidean distance by default).
The most common label among those neighbours becomes the predicted class.
k (number of neighbours):
Small k = very local, sensitive to noise
Large k = smoother, more general decision
Distance metric: Often Euclidean (straight-line), but others like Manhattan (like a grid) can be used.
Not really. It’s lazy learning:
No training or model-building phase.
All the “thinking” happens at the time of prediction.
This makes KNN simple but computationally expensive for large datasets.