86
$\begingroup$

I know that k-means is unsupervised and is used for clustering etc and that k-NN is supervised. But I wanted to know concrete differences between the two?

$\endgroup$
106
$\begingroup$

These are completely different methods. The fact that they both have the letter K in their name is a coincidence.

K-means is a clustering algorithm that tries to partition a set of points into K sets (clusters) such that the points in each cluster tend to be near each other. It is unsupervised because the points have no external classification.

K-nearest neighbors is a classification (or regression) algorithm that in order to determine the classification of a point, combines the classification of the K nearest points. It is supervised because you are trying to classify a point based on the known classification of other points.

$\endgroup$
  • 6
    $\begingroup$ I think there is more similarity than this guy is giving credit. They both use distance methods to cluster and classify inputs respectively. This is often why they are taught together, and why dimensionality issues are discussed in relation to them. Various distance methods can be applied to both. There are in fact a lot of similarities. $\endgroup$ – eljusticiero67 Oct 16 '18 at 21:04
  • $\begingroup$ @eljusticiero67 of course they are used to classify inputs, this is mentioned by OP. And most classical learning methods are distance based, so this is also not surprising. Note that the OP was interested in the differences. Also I understood it as though OP was implying there might be similarity due to the K in both names. $\endgroup$ – Bitwise Oct 17 '18 at 6:57
12
$\begingroup$

As noted by Bitwise in their answer, k-means is a clustering algorithm. If it comes to k-nearest neighbours (k-NN) the terminology is a bit fuzzy:

  • in the context of classification, it is a classification algorithm, as also noted in the aforementioned answer

  • in general it is a problem, for which various solutions (algorithms) exist

So in the first context, saying "k-NN classifier" can actually mean various underlying concrete algorithms that solve the k-NN problem, and their result is interpreted for the classification purpose.

These are two different things but you might find it interesting that k-means algorithm is one of various possible methods for solving the k-NN problem (Marius Muja and David G. Lowe, "Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration", in International Conference on Computer Vision Theory and Applications (VISAPP'09), 2009 PDF)

$\endgroup$
0
$\begingroup$

You can have a supervised k-means. You can build centroids (as in k-means) based on your labeled data. Nothing stops you. If you want to improve this, Euclidean space and Euclidean distance might not provide you the best results. You will need to choose your space (could be Riemannian space for example) and define the distance between points (and even define a "point"). The last two are topics of research and they also depend on the type (properties) of data (signal) you have.

$\endgroup$
-2
$\begingroup$

K-means can create the cluster information for neighbour nodes while KNN cannot find the cluster for a given neighbour node.

$\endgroup$
-2
$\begingroup$

k Means can be used as the training phase before knn is deployed in the actual classification stage. K means creates the classes represented by the centroid and class label ofthe samples belonging to each class. knn uses these parameters as well as the k number to classify an unseen new sample and assign it to one of the k classes created by the K means algorithm

$\endgroup$

protected by kjetil b halvorsen Nov 23 '17 at 12:46

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.