1
$\begingroup$


I'm working on my first clustering assignement and I've ran K-Means, Spectral clustering, Hierarchical clustering and Mini-Batch K-Means on same data and received the exact same results (cluster sizes, Calinski-Harabasz criterion, Silhouette). Does anyone have any idea, why would this be? Could this be only dependant on my dataset (6 variables, 2400 entities)?
Initialization of algorithms:

k_means = KMeans(init='k-means++', n_clusters=2)

mini_batch_k_means = MiniBatchKMeans(init='k-means++', n_clusters=2,n_init = 10, max_no_improvement = 10)

hierarchical_clustering = AgglomerativeClustering(n_clusters=2, linkage='ward')

spectral_clustering = SpectralClustering(n_clusters=2)

Thank you for your answers!

EDIT: Correction, cluster sizes, Silhouette and CH index are the same, but not the actual clusters. The actual clusters are just extremely similar with some differences on borders. As suggested I projected my data points to 3D plane (with PCA), but as you can see from the image below, there is no clear distinction of the two clusters. Projection of data points on 3D plane - PCA

Curiously enough, when I made a decision tree on clustered data (for better visualization), I noticed that the decision tree only uses one variable to classify to which cluster does the data belong.

$\endgroup$
  • $\begingroup$ Correction, cluster sizes, Silhouette and CH index are the same, but not the actual clusters. The actual clusters are just extremely similar. $\endgroup$ – Luka Žontar Dec 8 '19 at 0:21
  • $\begingroup$ Welcome to the site. Regarding your correction, it's best to edit this into your question, rather than post it as a comment. This way, the question will be self-contained and people reading it can understand the situation more easily. $\endgroup$ – user20160 Dec 8 '19 at 3:35
  • $\begingroup$ If your data has very obvious, well separated clusters, all the methods tens to work. Differences become more pronounced when you have difficult data with noise, non-convex clusters, overlapping clusters, large size and density differences of clusters, etc. $\endgroup$ – Has QUIT--Anony-Mousse Dec 8 '19 at 8:22
  • $\begingroup$ I edited the question, thank you for your suggestion! @Anony-Mousse thank you for your answer! $\endgroup$ – Luka Žontar Dec 8 '19 at 9:02
  • $\begingroup$ When the scores are "the same", maybe they are equally bad? A Silhouette of 0.2 does not mean much, if anything... But maybe also your evaluation code is wrong. $\endgroup$ – Has QUIT--Anony-Mousse Dec 8 '19 at 23:49
2
$\begingroup$

It does indeed depend on the dataset. Not knowing the dataset it is difficult to know the exact reason, but most probably the 2 clusters you're trying to estimate do indeed exist in your data and are clearly distinguishable.

To get an insight into whether this is the case, you could try to project the data points onto the 2D or 3D plane (e.g. via PCA or t-SNE) and visually inspect the data.

The difference in clusters you mention may be due to bordering data points fluctuating between both clusters depending on the algorithm initialization. You could run multiple times each algorithm and get an estimate of the average values for your metrics. Another option is to plot different runs of the algorithm (after projecting on 2D) to check for consistency across runs.

In summary, yes. You are obtaining almost identical results because of your dataset but that's not the rule. Distance, density and hierarchical clustering algorithms tend to produce different results (also because of the hyperparameters).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks! I edited my question and added projection to 3D plane and a detail that might help with resolving this question. To comment your answer, most probably all the algorithms estimate the same clusters, which might indeed suggest that they are there. The reason why I asked the question is because I can't see the difference between them. Furthermore, if I run the algorithms multiple times I always get the exact same cluster sizes, Silhouette and CH index. $\endgroup$ – Luka Žontar Dec 8 '19 at 9:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.