# How to interpret dendrogram height for clustering by correlation

Given the following data frame:

df <- data.frame(x1 = c(26, 28, 19, 27, 23, 31, 22, 1, 2, 1, 1, 1),
x2 = c(5, 5, 7, 5, 7, 4, 2, 0, 0, 0, 0, 1),
x3 = c(8, 6, 5, 7, 5, 9, 5, 1, 0, 1, 0, 1),
x4 = c(8, 5, 3, 8, 1, 3, 4, 0, 0, 1, 0, 0),
x5 = c(1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0),
x6 = c(2, 3, 1, 0, 1, 1, 3, 37, 49, 39, 28, 30))


Such that

> df
x1 x2 x3 x4 x5 x6
1  26  5  8  8  1  2
2  28  5  6  5  1  3
3  19  7  5  3  1  1
4  27  5  7  8  1  0
5  23  7  5  1  1  1
6  31  4  9  3  0  1
7  22  2  5  4  1  3
8   1  0  1  0  0 37
9   2  0  0  0  0 49
10  1  0  1  1  0 39
11  1  0  0  0  0 28
12  1  1  1  0  0 30


I would like to group these 12 individuals using hierarchical clusters, and using the correlation as the distance measure. So this is what I did:

clus <- hcluster(df, method = 'corr')


And this is the plot of clus:

This df is actually one of 69 cases I'm doing cluster analysis on. To come up with a cutoff point, I have looked at several dendograms and played around with the h parameter in cutree until I was satisfied with a result that made sense for most cases. That number was k = .5. So this is the grouping we've ended up with afterwards:

> data.frame(df, cluster = cutree(clus, h = .5))
x1 x2 x3 x4 x5 x6 cluster
1  26  5  8  8  1  2       1
2  28  5  6  5  1  3       1
3  19  7  5  3  1  1       1
4  27  5  7  8  1  0       1
5  23  7  5  1  1  1       1
6  31  4  9  3  0  1       1
7  22  2  5  4  1  3       1
8   1  0  1  0  0 37       2
9   2  0  0  0  0 49       2
10  1  0  1  1  0 39       2
11  1  0  0  0  0 28       2
12  1  1  1  0  0 30       2


However, I am having trouble interpreting the .5 cutoff in this case. I've taken a look around the Internet, including the help pages ?hcluster, ?hclust and ?cutree, but with no success. The farthest I've become to understanding the process is by doing this:

First, I take a look at how the merging was made:

> clus$merge [,1] [,2] [1,] -9 -11 [2,] -8 -10 [3,] 1 2 [4,] -12 3 [5,] -1 -4 [6,] -3 -5 [7,] -2 -7 [8,] -6 7 [9,] 5 8 [10,] 6 9 [11,] 4 10  Which means everything started by joining observations 9 and 11, then observations 8 and 10, then steps 1 and 2 (i.e., joining 9, 11, 8 and 10), etc. Reading about the merge value of hcluster helps understand the matrix above. Now I take a look at each step's height: > clus$height
[1] 1.284794e-05 3.423587e-04 7.856873e-04 1.107160e-03 3.186764e-03 6.463286e-03
6.746793e-03 1.539053e-02 3.060367e-02 6.125852e-02 1.381041e+00