84
votes
Choosing the right linkage method for hierarchical clustering
Methods overview
Short reference about some linkage methods of hierarchical agglomerative cluster analysis (HAC).
Basic version of HAC algorithm is one generic; it amounts to updating, at each step, ...
- 52.6k
63
votes
Accepted
How to select a clustering method? How to validate a cluster solution (to warrant the method choice)?
Often they say that there is no other analytical technique as strongly of the "as you sow you shall mow" kind, as cluster analysis is.
I can imagine of a number dimensions or aspects of "rightness" ...
- 52.6k
29
votes
How to interpret the dendrogram of a hierarchical cluster analysis
I had the same questions when I tried learning hierarchical clustering and I found the following pdf to be very very useful.
http://www.econ.upf.edu/~michael/stanford/maeb7.pdf
Even if Richard is ...
- 391
26
votes
Accepted
Using correlation as distance metric (for hierarchical clustering)
Requirements for hierarchical clustering
Hierarchical clustering can be used with arbitrary similarity and dissimilarity measures. (Most tools expect a dissimilarity, but will allow negative values - ...
16
votes
How to understand the drawbacks of Hierarchical Clustering?
Whereas $k$-means tries to optimize a global goal (variance of the clusters) and achieves a local optimum, agglomerative hierarchical clustering aims at finding the best step at each cluster fusion (...
- 3,928
14
votes
Accepted
Do I need to remove duplicate objects for cluster analysis of objects?
It changes the results. With k-means this should be straightforward to see: the mean of 0, 0 and 1 is different from 0 and 1. Usually this will also be the case for hierarchical clustering, but it ...
- 6,959
14
votes
Clustering -- Intuition behind Kleinberg's Impossibility Theorem
One way or another, every clustering algorithm relies on some notion of “proximity” of points. It seems intuitively clear that you can either use a relative (scale-invariant) notion or an absolute (...
14
votes
How to understand the drawbacks of Hierarchical Clustering?
Scalability
$k$ means is the clear winner here. $O(n\cdot k\cdot d\cdot i)$ is much better than the $O(n^3 d)$ (in a few cases $O(n^2 d)$) scalability of hierarchical clustering because usually both $...
12
votes
Accepted
Why are the cluster analysis results using raw data the same as the ones using PCA scores?
This is because PCA scores are simply original data in a rotated coordinate frame.
Below on the left I show some example 2D data (100 points in 2D) and on the right the corresponding PCA scores. The ...
- 95.1k
10
votes
Hierarchical clustering of correlation matrix
To apply most hierarchical clustering/heatmap tools you'll need to convert your correlation matrix into a distance matrix (ie 0 is close together, higher is further apart). This blog post covers some ...
- 2,157
10
votes
How to select a clustering method? How to validate a cluster solution (to warrant the method choice)?
There are mostly red flag criteria. Properties of data that tell you that a certain approach will fail for sure.
if you have no idea what your data means stop analyzing it. you are just guessing ...
9
votes
How to understand the drawbacks of Hierarchical Clustering?
I just wanted to add to the other answers a bit about how, in some sense, there is a strong theoretical reason to prefer certain hierarchical clustering methods.
A common assumption in cluster ...
- 191
9
votes
Accepted
What is the interpretation of eps parameter in DBSCAN clustering?
Epsilon is the local radius for expanding clusters. Think of it as a step size - DBSCAN never takes a step larger than this, but by doing multiple steps DBSCAN clusters can become much larger than eps....
8
votes
How to interpret dendrogram height for clustering by correlation
Recall that in hierarchical clustering, you must define a distance metric between clusters. For example, in hierarchical average linkage clustering (probably the most popular option), the distance ...
- 6,459
8
votes
Do I need to remove duplicate objects for cluster analysis of objects?
If you remove duplicates, you need to add weights to your data instead, otherwise the result may change (except for single-linkage clustering, I guess).
If your data set has few duplicates, this will ...
7
votes
Clustering -- Intuition behind Kleinberg's Impossibility Theorem
This is the intuition I came up with (a snippet from my blog post here).
A consequence of the richness axiom is that we can define two different distance functions, $d_1$ (top left) and $d_2$ (bottom ...
- 2,416
6
votes
Accepted
Is it necessary to normalize data for hierarchical clustering of mixed variables using complete linkage?
Transforming your data by subtracting the minimum from every value and dividing the differences by the range is often called normalizing. The transformed data will lie within the interval $[0, 1]$.
...
6
votes
What is the interpretation of eps parameter in DBSCAN clustering?
The meaning of $\epsilon$ is that of the neighbourhood size. The neighbourhood of a point $p$, denoted by $N_{\epsilon}(p)$, is defined as the $N_{\epsilon}(p) = \{q \in D | dist(p,q) \leq \epsilon \}$...
- 36k
6
votes
Accepted
Conditional intraclass correlation (ICC) from a linear mixed model as evidence for test-retest reliability?
Yes, you can do this and interpret it as you think. I have read about such an interpretation in the second chapter of Sophia Rabe-Hesketh and Anders Skrondal's Multilevel and Longitudinal Modeling ...
- 3,337
5
votes
Accepted
Analysis of hierarchical clustered hospital data
A perspective from Gelman & Hill's Data Analysis Using Regression and Multilevel/Hierarchical Models may point in a helpful direction. G&H entirely eschew the terms 'fixed' and 'random', ...
- 2,117
5
votes
Validate dendrogram in cluster analysis: What is the meaning of cophenetic correlation coefficient?
The cophenetic correlation coefficient is defined as the linear correlation between the dissimilarities $d_{ij}$ between each pair of observations $(i,j)$ and their corresponding cophenetic distances $...
- 51
5
votes
Accepted
Mahalanobis distance in a hierarchical cluster analysis in SPSS
IBM advises against using the Mahalanobis' distance in clustering. See here.
In hierarchical clustering, you need to define the distance between the clusters (as they are formed) and the remaining ...
- 13.6k
5
votes
Can sub-optimality of various hierarchical clustering methods be assessed or ranked?
Only single-linkage is optimal. Complete-linkage fails, so your intuition was wrong there. ;-)
As a simple example, consider the one-dimensional data set 1,2,3,4,5,6.
The (unique) optimum solution ...
- 2,818
5
votes
Which unsupervised classification method to use next if hierarchical clustering gave bad results?
I loaded your data into R and applied hierarchical clustering with Ward's method, which gave 3 clean cut clusters for your stations (Fig.1). Then I applied Principal Component Analysis on the scaled ...
- 2,447
5
votes
Choosing the right linkage method for hierarchical clustering
The correlation between the distance matrix and the cophenetic distance is one metric to help assess which clustering linkage to select. From ?cophenetic:
It can ...
- 230
5
votes
Is there any situation where PCA performs better than SVD?
PCA and SVD are not comparable. In short, SVD is a technique that one can use to compute the principal components in a PCA. It is possible to find the principal components without using SVD by ...
- 368
5
votes
Accepted
Motivation for Ward's definition of error sum of squares (ESS)
\begin{align}
\operatorname{Var}(\vec x) \propto \sum_{i=1}^n(x_i - \bar x)^2 &= \sum_i x_i^2 - 2\bar x \sum_ix_i + n \bar x^2 \\
&= \sum_i x_i^2 - n \bar x^2 = \sum_i x_i^2 - \frac 1n \left(\...
- 18.7k
5
votes
Does k-means have any advantages over HDBSCAN expect for runtime?
Randomization can be valuable. You can run k-means several times to get different possible clusters, as not all may be good. With HDBSCAN, you will always get the same result again.
Classifier: k-...
5
votes
Accepted
Best practices in the selection of distance metric and clustering methods for gene expression data
This will probably not be the answer you want or expect, but this is how I see these things.
Clustering problem
Clustering, to a degree, is almost always a subjective procedure. You decide how you ...
- 4,387
4
votes
Is there an advantage to squaring dissimilarities when using Ward clustering?
From the Conclusion of Murtaugh, F. & Legendre, P. (2011). Ward's Hierarchical Clustering Method: Clustering Criterion and Agglomerative Algorithm, ArXive:1111.6285v2 (pdf):
Two algorithms, ...
- 41
Only top scored, non community-wiki answers of a minimum length are eligible