I would like to ask a question about Gower similarity/dissimilarity index. Is it ok to use the Gower dissimilarity measure with Ward linkage clustering? I was reading that the Gower similarity index should not be used with Ward linkage because the index is not metric. I was wondering if this is only the case for the similarity and not for the dissimilarity index that can also handle odrinal variables?!
$\begingroup$
$\endgroup$
1
-
$\begingroup$ maybe helpful: stats.stackexchange.com/questions/9342/… $\endgroup$– steffenFeb 3, 2012 at 11:05
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
3
- Gower dissimilarity is just 1 minus Gower similarity, $1-GS$. So, they are "the same", and limitations of one are the limitations of the other.
- Ward clustering computes cluster centroids and in order for those to be geometrically "real" it demands (squared) euclidean distances as its input. Euclidean distance is metric. Not every metric distance is euclidean. Thus, not every metric distance is correct for Ward. Still, in practice, metric distances that are not euclidean distance could be used with Ward method heuristically. Non-metric distances - they are not recommended with Ward at all.
- By origin, Gower dissimilarity is non-euclidean and non-metric (even when all variables to compute it had been interval, Gower index will be closer to Manhattan distance, not euclidean distance), so you cannot use Ward.
- However, geometrically, a concrete matrix of Gower dissimilarity could happen to be close to euclidean distance, and then you may be licensed using Ward (just with these specific data!). To check if a dissimilarity matrix is (or is close to be) euclidean or not geometrically, one should double-center it and inspect the eigenvalues of the resultant matrix. The smaller is the sum of negative eigenvalues relative to the sum of positive ones, the closer is the dissimilarities to euclidean distances. But even in this occured case using Ward with Gower distance is purely heuristic. Read about the properties of double centered scalar product matrix here.
- Gower dissimilarity defined as $\sqrt {1-GS}$ is actually a Euclidean distance (therefore metric, automatically) when no specially processed ordinal variables were used. After double-centering the matrix of this dissimilarity, the obtained scalar product (Gram) matrix has no negative eigenvalues at all (therefore it spans euclidean space with full convergence). So, just use this version of the dissimilarity if you want to use methods demanding euclidean space and if taking square root is an acceptable transform for your study settings.
-
$\begingroup$ ok, I see. thank you very much for your answer. What would you advise me? I have around 10 dummy-variables, that are correlated, one ordinal variable on a likert scale with 7 types and around four metric variables (they can only have values between 0 and 100). I have 2500 observations. I think one alternative is gower index, combined with complete,single or average linkage. I also heard about two-step clustering, but I heard that the variables should be independent for this?! $\endgroup$ Feb 6, 2012 at 9:20
-
$\begingroup$ I think you're justified to try both, Gower+hierarchical and two-step. Generally, hierarchical clustering is not recommended with more than about 300-500 objects, because it is greedy algorithm and can give suboptimal solutions on later steps if the number of steps is large; but I tentatively suppose this does not pertain to single or complete methods (average -??). Two-step cluster indeed assumes uncorrelated variables, though it is quite robust in this respect. I allows metrical and nominal variables, so you'll probably have to treat your ordinal one as metrical and binary ones as nominal. $\endgroup$– ttnphnsFeb 6, 2012 at 11:12
-
$\begingroup$ I have the same question as the OP - a colleague and I built a Gower dissimilarity distance matrix for member-level health care data containing binary & ratio variables, then successfully clustered the distance matrix using Ward's minimum variance method. I understand the limitations of using Ward with non-Euclidean distances, however average linkage was producing too many single observation clusters, which Ward avoided. $\endgroup$– RobertFMay 10, 2017 at 15:51
$\begingroup$
$\endgroup$
8
I'm a little puzzled by the answer above because of the following paragraph from Gower (1971):
- POSITIVE SEMI-DEFINITE PROPERTY OF THE SIMILARITY MATRIX
With n individuals, the n X n matrix S can be formed whose element Sij is the similarity, as described in section 2, between individuals i and j. We often require to represent the n individuals of a sample as a set of points in Euclidean space. Gower (1966) has discussed this problem and shown that a convenient representation can be obtained by taking the distance between the ith and jth individuals as proportional to (1 - Sij)^(1/2). The coordinates of points with these distances are the elements of the latent vectors of S scaled so that their sums of squares equal the latent roots. Thus to get a real Euclidean representation with distances (1 - Sij)^(1/2) it is sufficient for S to be positive semi-definite (p.s.d.). It is shown in the Appendix that when there are no missing values S is p.s.d.
By my read of this, although it's true that the output of a Gower's distance function (e.g. D=as.matrix(daisy(data, metric="gower"))) isn't Euclidean on its own, if there is no missing data, a Euclidean transform should be obtainable -- and if there is missing data, but you find that S=1-D is positive semidefinite (e.g. using matrixcalc::is.positive.semi.definite(S)), then you should still be able to do that same transform.
(Incidentally, I think whether Ward's method wants Euclidean distances or squared Euclidean distances depends on the implementation. Based on the documentation, I believe the version in agnes() wants Euclidean distances.)
-
$\begingroup$ The citation, which is about "Gower formula" $d=\sqrt{(1-s)}$ has nothing to do with "Gower similarity measure". Just Gower said/invented many different useful things. The cited formula says that if a similarity matrix is gramian (= positive semidefinite) then the formula (it is actually the trigonometric cosine theorem) converts it exactly, without loss, into euclidean distances. $\endgroup$– ttnphnsDec 6, 2017 at 18:01
-
$\begingroup$ Next, it is quite strange to compute Gower similatity $s$ from "gower distance" because the latter is what is being computed from Gower Similarity measure as d=1-s. Gower s is not an angular euclidean similarity and its matrix is often non-gramian. $\endgroup$– ttnphnsDec 6, 2017 at 18:05
-
$\begingroup$ Wards's linkage method is based on locating centroids of clusters and computing deviations from them. It needs therefore euclidean distances. With other metric distances, Ward won't be a mathematically "exact" method, only heuristic. $\endgroup$– ttnphnsDec 6, 2017 at 18:08
-
$\begingroup$ I beg to differ on relevance - this is an excerpt from the paper cited within
daisy, namely: Gower, J. C. (1971) A general coefficient of similarity and some of its properties, Biometrics 27, 857–874. The "similarity, as described in section 2," he is referring to is his own -- so he is making a claim that applies to his own metric. (I backformed this S from D because S is not output bydaisy.) Of course, real world data have missingness, so this neat result may not apply (as it did not in the dataset that led me here - S was non-psd). But I think your "no" was misleadingly strong. $\endgroup$– ErinMcJDec 6, 2017 at 21:05 -
$\begingroup$ You are right that what input is required in Ward - squared or nonsquared Euclidean d - depends on the implementation. As for citation, I didn't say it's irrelevant. I said, one should not confuse Gower similarity measure with "Gower's" s-to-d convertion formula only because they bear same name "Gower". $\endgroup$– ttnphnsDec 6, 2017 at 21:07