2
$\begingroup$

When i perform a Hotelling's $T^2$ test on a dataset, it stated that there is strong evidence that the mean vectors of the two groups differ. However, when I create a dendrogram, I got:

dendrogram

where number 1- 17 suppose to be group one and number 18-32 be group 2. But the dendrogram shows many overlap. Why?

dataset: (http://www.nbi.dk/~petersen/Teaching/Stat2010/Data_TibetanSkulls.txt)

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Because you're comparing the dendogram for hierarchical cluster analysis (HCA) based on Euclidean distance with a multivariate means test -- which are by no means similar comparisons. You can always have noisy and jumpy data with outliers present, which will cause jumpyness in the dendogram results.

With multiple features, it's always difficult to show why multivariate-based averages are different across groups, since it's not that straightforward. Below is a 2D score plot for a variety of unsupervised methods: and Laplacian Eigenmaps (LEM) shows separation between the two classes. NMF does as well, but not as good. I couldn't get any HCA results using various distance metrics which partition the objects into two separate clusters, mostly due to the overlap of feature values across the two groups.

enter image description here

Also, below is a 3D run using LEM:

enter image description here

Improve this answer
$\endgroup$
2
  • $\begingroup$ So the Hotelling's $T^2$ test tells us their mean are different, but the HCA tells us the differences (not mean differences) between the two groups? Let's say we gnore the $T^2$ test and just look at the HCA. Does the HCA state that the two groups are classified wrong? $\endgroup$
    – GarlicSTAT
    Commented Mar 2, 2020 at 1:33
  • 1
    $\begingroup$ Hotelling's is not Euclidean distance, so they're not really comparable. There could almost be an infinite number of way the dendogram looks when Hotelling's test is significant. HCA also does not classify, since it's an unsupervised method (truth table is not used). The question regarding "classified wrong" requires use of a supervised classifier, like regression, linear discriminant analysis, logistic regression, decision tree classifier, support vector machines, neural network, etc. $\endgroup$
    – user32398
    Commented Mar 2, 2020 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.