It seems like a trivial extension of k-means clustering if you have labelled training data is to assign each centroid a label, and given a new piece of test data to classify it to the label corresponding to the centroid with the closest distance.

However, I could not find much online about the (ab)use of k-means clustering for this purpose, and people advise using just k-nearest neighbors or SVMs directly.

Why would (ab)using k-means clustering via this relatively straightforward method not yield good results?

  • $\begingroup$ How would it work? There is no outcome variable in K-means, often the clusters that K-means finds does not corrospond to the labels you would like classification for $\endgroup$ – Repmat Apr 5 '18 at 8:04
  • $\begingroup$ Hm I found this which shows that it's at least possible, although it doesn't discuss the positives/negatives of doing so: onlinecourses.science.psu.edu/stat857/node/113 $\endgroup$ – 1110101001 Apr 5 '18 at 21:15
  • $\begingroup$ I think that article sums up pretty well why you shouldnt do that, its tedious and cumbersome. Why not just use a method that directly attemps to solve the problem you have? $\endgroup$ – Repmat Apr 6 '18 at 11:13

Where is the benefit of doing this?

A cluster found by KMeans may contain many different labels, so you decrease quality!

Just so nearest-neighbor classification.

But if you look into kNN classification literature, you will find different ways of reducing the "training" set. I am certain there is at least one paper suggesting to use k-means with a very large k to reduce redundancy in your training data. But you need to smartly handle clusters that contain more than one label.


Kmeans Clustering is a Cluster Algorithm to separate your data into different clusters. Of course you can assign each cluster a representing class. When you got an additional unknow point you calculate the distance to the nearest centroid (cluster center) and add this point to this cluster/class.

What does it mean is, that you would use for this classification task a k nearest neighbor approach, where you just look at the centers of the clusters as possible neighbors.

So what we've got is a pretty simple classifier and of course there are better ways to classify data.


I would say that k-means could be advised for classifitation following a different approach:

Let $C$ be the number of classes and $K$ the number of clusters. Now, follow these steps:

  1. Apply K-means clustering to the training data in each class seperately, using $K$ clusters per class.
  2. Assign a class label to each of the $C*K$ clusters.
  3. Classify observation $x$ to the class of the closest cluster.

I saw it on "The Elements of Statistical Learning". I changed the notation a little bit. It seems like a nice approach for classification that reduces data observations by using clusters.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.