It is said that kMeans clustering works as long as we don't have clusters of differing

  1. sizes,
  2. densities,
  3. and non-spherical shapes

I understand how one might check the sizes and densities of data points, but I am unsure how one would check for non-spherical clusters, especially if one merely has a conceptual understanding of a dataset.

So, Q1. Since most people with real world data sets would not know if they have non-spherical clusters, would it not be a mistake to apply kMeans?

Q2. What would be a more suitable choice of a clustering algorithm if we only want to partition our data set, given we don't know the shape of our data? By partition, I refer to partitioning cluster algorithms such as kMeans and kMedoids.

Q3. What mind map would experienced clustering users have when they intend to apply partitioning cluster algorithms?

For example, the following is mine, but I am inexperienced in clustering:

  1. check if the data is comprised of continuous, or continous + discrete variables
  2. pick euclidean/manhattan/mahalanobis distance measure for former, or gower for latter
  3. perhaps standardise first and then apply the distance measure/or correlation if a similarity matrix is sought
  4. apply a partitioning cluster algorithm; kMeans if we know our data shape and outliers are not a problem. kMedoids if we don't want to be affected by outliers (although the definition of a centroid/prototype as found by kMeans seems to change if we apply kMedoids, i.e. a centroid in kMeans is understandably a prototype of each cluster, while in kMedoids this centroid is simply another data point and not a prototype)
  5. validate cluster results (using the Elbow test/Dunn index/Silhouette Coeff if focusing only on internal measures) and also check for clustering tendency

Am I missing something important in the above?

Many thanks, and please pardon the ignorance.

  • 1
    $\begingroup$ Q2. If you want only to partition (somehow) data, why use clustering at all? You use "partition" also in Q3.4. It is unclear what you mean by partition in your question: 1) any partition, 2) any clustering, or 3) partitioning methods of clustering (such as K-means, but not at all hierarchical agglomerative). $\endgroup$ – ttnphns Mar 5 '14 at 14:51
  • $\begingroup$ Apologies, I forgot about the general meaning of a partition. What I meant was a partitioning cluster algorithm - so no hierarchical clustering, only those cluster algroithms that split data into flat partitions. $\endgroup$ – info_seeker Mar 5 '14 at 14:53
  • $\begingroup$ Then please edit your question to make it clear. AND, it would be nice if you mention the partitioning methods you are interested in or you know. $\endgroup$ – ttnphns Mar 5 '14 at 14:55
  • $\begingroup$ I hope now it is sufficient. Apologies. I understand it is a wide topic and some questions may seem too general - bear with me please. $\endgroup$ – info_seeker Mar 5 '14 at 15:03

If you are less interested in finding structure of your data then just "simplifying" your data - see vector quantization - then methods such as k-means can work okay.

However, often you don't even need k-means.

Just pick random points as cluster centers and partition your data accordingly.

Or make data bubbles / canopies. Overlapping groups of points, with a distance threshold.

There are many methods which mostly compress your data (reduce it to a number of representative points), which do not actually try to discover some complex structure or optimize some mathematical property.

The main question is: is this still "clustering"? Or is it more of a "data simplification", "vector quantization", "sampling" method?

| cite | improve this answer | |
  • $\begingroup$ Thank you for replying - great to learn from people like yourself. Please pardon my delayed response. I originally wanted to ask for something like a step-by-step procedure to determine an appropriate clustering technique. But as I have read answers here (and some books), clustering is a lot more complex and nuanced than this. And I'm still trying to answer the question you put forth... $\endgroup$ – info_seeker Apr 7 '14 at 18:49
  • 1
    $\begingroup$ The more I see, it is about compressing the data and selecting representative samples. Which is why quantization methods like k-means often work very well on data where they are ''not supposed to work'' (because of outliers etc.) - if we only need a very rough solution, they are good enough. $\endgroup$ – Has QUIT--Anony-Mousse Apr 7 '14 at 23:04
  • $\begingroup$ Ah, I see. I was also wondering how come k-means is so widely applied-for example, in customer profiling, where all sorts of categorical, binary, and continuous variables occur, and it is not clear what it means to compute Euclidean distance between variables such as amount of phone usage, age, salary, etc. Neither is it certain that such data will not have outliers, different shapes, densities, etc. But now as I understand k-means more, I will be more careful in my analysis, hopefully, and also look for algorithms detecting structure (in my understanding hierarchical clustering)...Thank you! $\endgroup$ – info_seeker Apr 8 '14 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.