On internet and in articles KNN ist mostly described as supervised algorithm. But recently I have find also few articles where it is mentioned as unsupervised algorithm.I cannot find articles that are reviewing/comparing both of them. Can you recommend someliterature/articles where I could read more about it? Can you explain how can KNN be both? What are the differences?
3
-
$\begingroup$ sklearn.neighbors.NearestNeighbors - knn for unsupervised learning? $\endgroup$– user2974951Commented Jun 9, 2022 at 10:10
-
$\begingroup$ Sklearn: unsupervised knn vs k-means $\endgroup$– user2974951Commented Jun 9, 2022 at 10:10
-
$\begingroup$ K-nearest neighbor supervised or unsupervised machine learning? $\endgroup$– user2974951Commented Jun 9, 2022 at 10:12
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The $k$-nearest neighbors supervised learning algorithm works by taking a point, calculating the distance in the feature space (predictors) between that point and all other points, determining the $k$ points that are closest, and using the labels ($y$) on those points to make a prediction. That prediction might be an average in order to do a regression. It might be the category with the most representation among those $k$-nearest neighbors. It might be a probability based on the relative number of categories represented in those $k$-nearest neighbors.
The key point, however, is that there is a feature space for calculating distances and then a target space of the variable(s) being predicted. This is exactly a supervised problem.
One unsupervised machine learning clustering algorithm is called “k-means”, which you notice has a similar but distinct name. The gist here is that the distance between points and the cluster mean are calculated in order to assign points to clusters. However, there is no target space, and this is not a supervised method.
A place where $k$-nearest neighbors could be used in an unsupervised way if if you just want to know the $k$-nearest neighbors in the feature space. This could be used, for instance, for some kind of matching procedure.
