84 votes

Choosing the right linkage method for hierarchical clustering

Methods overview Short reference about some linkage methods of hierarchical agglomerative cluster analysis (HAC). Basic version of HAC algorithm is one generic; it amounts to updating, at each step, ...
user avatar
  • 52.6k
63 votes
Accepted

How to select a clustering method? How to validate a cluster solution (to warrant the method choice)?

Often they say that there is no other analytical technique as strongly of the "as you sow you shall mow" kind, as cluster analysis is. I can imagine of a number dimensions or aspects of "rightness" ...
user avatar
  • 52.6k
29 votes

How to interpret the dendrogram of a hierarchical cluster analysis

I had the same questions when I tried learning hierarchical clustering and I found the following pdf to be very very useful. http://www.econ.upf.edu/~michael/stanford/maeb7.pdf Even if Richard is ...
user avatar
  • 391
26 votes
Accepted

Using correlation as distance metric (for hierarchical clustering)

Requirements for hierarchical clustering Hierarchical clustering can be used with arbitrary similarity and dissimilarity measures. (Most tools expect a dissimilarity, but will allow negative values - ...
user avatar
16 votes

How to understand the drawbacks of Hierarchical Clustering?

Whereas $k$-means tries to optimize a global goal (variance of the clusters) and achieves a local optimum, agglomerative hierarchical clustering aims at finding the best step at each cluster fusion (...
user avatar
  • 3,928
14 votes
Accepted

Do I need to remove duplicate objects for cluster analysis of objects?

It changes the results. With k-means this should be straightforward to see: the mean of 0, 0 and 1 is different from 0 and 1. Usually this will also be the case for hierarchical clustering, but it ...
user avatar
  • 6,959
14 votes

Clustering -- Intuition behind Kleinberg's Impossibility Theorem

One way or another, every clustering algorithm relies on some notion of “proximity” of points. It seems intuitively clear that you can either use a relative (scale-invariant) notion or an absolute (...
user avatar
14 votes

How to understand the drawbacks of Hierarchical Clustering?

Scalability $k$ means is the clear winner here. $O(n\cdot k\cdot d\cdot i)$ is much better than the $O(n^3 d)$ (in a few cases $O(n^2 d)$) scalability of hierarchical clustering because usually both $...
user avatar
12 votes
Accepted

Why are the cluster analysis results using raw data the same as the ones using PCA scores?

This is because PCA scores are simply original data in a rotated coordinate frame. Below on the left I show some example 2D data (100 points in 2D) and on the right the corresponding PCA scores. The ...
user avatar
  • 95.1k
10 votes

Hierarchical clustering of correlation matrix

To apply most hierarchical clustering/heatmap tools you'll need to convert your correlation matrix into a distance matrix (ie 0 is close together, higher is further apart). This blog post covers some ...
user avatar
10 votes

How to select a clustering method? How to validate a cluster solution (to warrant the method choice)?

There are mostly red flag criteria. Properties of data that tell you that a certain approach will fail for sure. if you have no idea what your data means stop analyzing it. you are just guessing ...
user avatar
9 votes

How to understand the drawbacks of Hierarchical Clustering?

I just wanted to add to the other answers a bit about how, in some sense, there is a strong theoretical reason to prefer certain hierarchical clustering methods. A common assumption in cluster ...
user avatar
  • 191
9 votes
Accepted

What is the interpretation of eps parameter in DBSCAN clustering?

Epsilon is the local radius for expanding clusters. Think of it as a step size - DBSCAN never takes a step larger than this, but by doing multiple steps DBSCAN clusters can become much larger than eps....
user avatar
8 votes

How to interpret dendrogram height for clustering by correlation

Recall that in hierarchical clustering, you must define a distance metric between clusters. For example, in hierarchical average linkage clustering (probably the most popular option), the distance ...
user avatar
  • 6,459
8 votes

Do I need to remove duplicate objects for cluster analysis of objects?

If you remove duplicates, you need to add weights to your data instead, otherwise the result may change (except for single-linkage clustering, I guess). If your data set has few duplicates, this will ...
user avatar
7 votes

Clustering -- Intuition behind Kleinberg's Impossibility Theorem

This is the intuition I came up with (a snippet from my blog post here). A consequence of the richness axiom is that we can define two different distance functions, $d_1$ (top left) and $d_2$ (bottom ...
user avatar
  • 2,416
6 votes
Accepted

Is it necessary to normalize data for hierarchical clustering of mixed variables using complete linkage?

Transforming your data by subtracting the minimum from every value and dividing the differences by the range is often called normalizing. The transformed data will lie within the interval $[0, 1]$. ...
user avatar
6 votes

What is the interpretation of eps parameter in DBSCAN clustering?

The meaning of $\epsilon$ is that of the neighbourhood size. The neighbourhood of a point $p$, denoted by $N_{\epsilon}(p)$, is defined as the $N_{\epsilon}(p) = \{q \in D | dist(p,q) \leq \epsilon \}$...
user avatar
6 votes
Accepted

Conditional intraclass correlation (ICC) from a linear mixed model as evidence for test-retest reliability?

Yes, you can do this and interpret it as you think. I have read about such an interpretation in the second chapter of Sophia Rabe-Hesketh and Anders Skrondal's Multilevel and Longitudinal Modeling ...
user avatar
  • 3,337
5 votes
Accepted

Analysis of hierarchical clustered hospital data

A perspective from Gelman & Hill's Data Analysis Using Regression and Multilevel/Hierarchical Models may point in a helpful direction. G&H entirely eschew the terms 'fixed' and 'random', ...
user avatar
5 votes

Validate dendrogram in cluster analysis: What is the meaning of cophenetic correlation coefficient?

The cophenetic correlation coefficient is defined as the linear correlation between the dissimilarities $d_{ij}$ between each pair of observations $(i,j)$ and their corresponding cophenetic distances $...
user avatar
  • 51
5 votes
Accepted

Mahalanobis distance in a hierarchical cluster analysis in SPSS

IBM advises against using the Mahalanobis' distance in clustering. See here. In hierarchical clustering, you need to define the distance between the clusters (as they are formed) and the remaining ...
user avatar
  • 13.6k
5 votes

Can sub-optimality of various hierarchical clustering methods be assessed or ranked?

Only single-linkage is optimal. Complete-linkage fails, so your intuition was wrong there. ;-) As a simple example, consider the one-dimensional data set 1,2,3,4,5,6. The (unique) optimum solution ...
user avatar
5 votes

Which unsupervised classification method to use next if hierarchical clustering gave bad results?

I loaded your data into R and applied hierarchical clustering with Ward's method, which gave 3 clean cut clusters for your stations (Fig.1). Then I applied Principal Component Analysis on the scaled ...
user avatar
  • 2,447
5 votes

Choosing the right linkage method for hierarchical clustering

The correlation between the distance matrix and the cophenetic distance is one metric to help assess which clustering linkage to select. From ?cophenetic: It can ...
user avatar
  • 230
5 votes

Is there any situation where PCA performs better than SVD?

PCA and SVD are not comparable. In short, SVD is a technique that one can use to compute the principal components in a PCA. It is possible to find the principal components without using SVD by ...
user avatar
5 votes
Accepted

Motivation for Ward's definition of error sum of squares (ESS)

\begin{align} \operatorname{Var}(\vec x) \propto \sum_{i=1}^n(x_i - \bar x)^2 &= \sum_i x_i^2 - 2\bar x \sum_ix_i + n \bar x^2 \\ &= \sum_i x_i^2 - n \bar x^2 = \sum_i x_i^2 - \frac 1n \left(\...
user avatar
  • 18.7k
5 votes

Does k-means have any advantages over HDBSCAN expect for runtime?

Randomization can be valuable. You can run k-means several times to get different possible clusters, as not all may be good. With HDBSCAN, you will always get the same result again. Classifier: k-...
user avatar
5 votes
Accepted

Best practices in the selection of distance metric and clustering methods for gene expression data

This will probably not be the answer you want or expect, but this is how I see these things. Clustering problem Clustering, to a degree, is almost always a subjective procedure. You decide how you ...
user avatar
4 votes

Is there an advantage to squaring dissimilarities when using Ward clustering?

From the Conclusion of Murtaugh, F. & Legendre, P. (2011). Ward's Hierarchical Clustering Method: Clustering Criterion and Agglomerative Algorithm, ArXive:1111.6285v2 (pdf): Two algorithms, ...
user avatar
  • 41

Only top scored, non community-wiki answers of a minimum length are eligible