34
$\begingroup$

In my dataset we have both continuous and naturally discrete variables. I want to know whether we can do hierarchical clustering using both type of variables. And if yes, what distance measure is appropriate?

$\endgroup$
  • $\begingroup$ What software are you using? $\endgroup$ – rolando2 Sep 7 '11 at 16:56
  • $\begingroup$ @rolando2: I'm using R (hclus package). $\endgroup$ – Beta Sep 7 '11 at 17:11
  • 1
    $\begingroup$ Is there a reason for using hierarchical clustering? $\endgroup$ – suncoolsu Sep 7 '11 at 19:13
  • $\begingroup$ Na. I just asking this question for the sake of my knowledge. Can we do hierarchical when we have mixed data type. If yes, then why? If no, then why? $\endgroup$ – Beta Sep 7 '11 at 19:55
  • $\begingroup$ @user4278 Well, what's your goal in fine? Are you looking to identify clusters (of individuals)? $\endgroup$ – chl Sep 7 '11 at 20:52
45
$\begingroup$

One way is to use Gower similarity coefficient which is a composite measure$^1$; it takes quantitative (such as rating scale), binary (such as present/absent) and nominal (such as worker/teacher/clerk) variables. Later Podani$^2$ added an option to take ordinal variables as well.

The coefficient is easily understood even without a formula; you compute the similarity value between the individuals by each variable, taking the type of the variable into account, and then average across all the variables. Usually, a program calculating Gower will allow you to weight variables, that is, their contribution, to the composite formula. However, proper weighting of variables of different type is a problem, no clear-cut guidelines exist, which makes Gower or other "composite" indices of proximity pull ones face.

The facets of Gower similarity ($GS$):

  • When all variables are quantitative (interval) then the coefficient is the range-normalized Manhattan distance converted into similarity. Because of the normalization variables of different units may be safely used. You should not, however, forget about outliers. (You might also decide to normalize by another measure of spread than range.) Because of the said normalization by a statistic, such as range, which is sensitive to the composition of individuals in the dataset Gower similarity between some two individuals may change its value if you remove or add some other individuals in the data.
  • When all variables are ordinal, then they are first ranked, and then Manhattan is computed, as above with quantitative variables, but with the special adjustment for ties.
  • When all variables are binary (with an asymmetric significance of categories: "present" vs "absent" attribute) then the coefficient is the Jaccard matching coefficient (this coefficient treats when both individuals lack the attribute as neither match nor mismatch).
  • When all variables are nominal (also including here dichotomous with symmetric significance: "this" vs "that") then the coefficient is the Dice matching coefficient that you obtain from your nominal variables if recode them into dummy variables (see this answer for more).

(It is easy to extend the list of types. For example, one could add a summand for count variables, using normalized chi-squared distance converted to similarity.)

The coefficient ranges between 0 and 1.

"Gower distance". Without ordinal variables present (i.e. w/o using the Podani's option) $\sqrt{1-GS}$ behaves as Euclidean distance, it fully supports euclidean space. But $1-GS$ is only metric (supports triangular inequality), not Euclidean. With ordinal variables present (using the Podani's option) $\sqrt{1-GS}$ is only metric, not Euclidean; and $1-GS$ isn't metric at all. See also.

With euclidean distances (distances supporting Euclidean space), virtually any classic clustering technique will do. Including K-means (if your K-means program can process distance matrices, of course) and including Ward's, centroid, median methods of Hierarchical clustering. Using K-means or other those methods based on Euclidean distance with non-euclidean still metric distance is heuristically admissible, perhaps. With non-metric distances, no such methods may be used.

The previous paragraph talks about if K-means or Ward's or such clustering is legal or not with Gower distance mathematically (geometrically). From the measurement-scale ("psychometric") point of view one should not compute mean or euclidean-distance deviation from it in any categorical (nominal, binary, as well as ordinal) data; therefore from this stance you just may not process Gower coefficient by K-means, Ward etc. This viewpoint warns that even if a Euclidean space is present it may be granulated, not smooth (see related).


$^1$ Gower J. C. A general coefficient of similarity and some of its properties // Biometrics, 1971, 27, 857-872

$^2$ Podani, J. Extending Gower’s general coefficient of similarity to ordinal characters // Taxon, 1999, 48, 331-340

$\endgroup$
  • $\begingroup$ Thanks ttnphns! Can you please tell me is "Gower similarity coefficient" is integrated in hclus package in R? Is there any package in R that has this functionality. $\endgroup$ – Beta Sep 8 '11 at 16:22
  • $\begingroup$ I'm not R user so don't know, and don't think you'll find it in hclus. But if you Google "Gower similarity R" you are sure to find what you need! $\endgroup$ – ttnphns Sep 8 '11 at 16:28
  • $\begingroup$ user4278, Just in case you don't shun SPSS, I have a macro for Gower similarity on my web-page. It accepts weighting and missing data. $\endgroup$ – ttnphns Sep 9 '11 at 7:44
  • $\begingroup$ @user4278 About R specifically, start here: cran.r-project.org/web/views/Environmetrics.html $\endgroup$ – chl Sep 9 '11 at 9:57
  • $\begingroup$ @ttnphns: Thanks you! I've visited you website & it is a great resource for SPSS. But unfortunately I don't use SPSS too much. But I'll surely use it if I don't get anything in R.@chl: Thank you! But it's a huge list. I read somewhere that Daisy, in cluster package, has functionality of Gower similarity. $\endgroup$ – Beta Sep 9 '11 at 12:57
15
$\begingroup$

If you have stumbled upon this question and are wondering what package to download for using Gower metric in R, the cluster package has a function named daisy(), which by default uses Gower's metric whenever mixed types of variables are used. Or you can manually set it to use Gower's metric.

daisy(x, metric = c("euclidean", "manhattan", "gower"),
      stand = FALSE, type = list(), weights = rep.int(1, p))
$\endgroup$
  • 4
    $\begingroup$ Further to Zhubarb's answer, if you want the Gower distances between all pairs from two data sets, then see the R package StatMatch. $\endgroup$ – James Hirschorn Sep 6 '15 at 19:40

protected by kjetil b halvorsen Jun 26 '18 at 19:02

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.