I want to use hierarchical clustering to classify some ecological data (species abundances on different places), so I would like to use a Manhattan type distance that doesn't account for double absences (e.g. Soergel Distance). The problem is that my data is too big to compute a distance matrix, but I found that several sites have exactly the same composition. Thus, I'm considering to 'manually' group those equal composition sites and to start the clustering in the middle of the dendrogram with only one observation (row) for each unique combination.
I could ignore that there is more than one observation with the same composition, and consider only unique cases, but it doesn't seem right.
I am using R. The hclust function allows to specify the number of members of each group. However, in the help of the function it says:

Dissimilarities between clusters can be efficiently computed (i.e., without hclust itself) only for a limited number of distance/linkage combinations, the simplest one being squared Euclidean distance and centroid linkage

My question is if there is any other combination of distance/linkage methods that can be used in order to get the same results when starting in the middle of the tree. I've tried a few using the example from the function but none worked.

  • 1
    $\begingroup$ You can ignore "that there is more than one observation" hiding behind a distance matrix row/column with such methods like single or complete linkage, but not such as Ward or centroid. I've written a SPSS macro hieclu which can perform clustering when rows/columns are groups of objects, even not identical. Even if you are not interested to use it you could read its description which might partly answer your question (visit my web and get the zip called 'Clustering'). $\endgroup$
    – ttnphns
    Commented Feb 27, 2020 at 14:04
  • 1
    $\begingroup$ This is also partly relevant, in case: stats.stackexchange.com/q/120044/3277 $\endgroup$
    – ttnphns
    Commented Feb 27, 2020 at 14:07


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.