-1
$\begingroup$

What type of clustering methods are available for ordinal, nominal and ratio variables? Suppose I have one ordinal, one nominal and one ratio variable; is there a common clustering technique that can be applied on the data set?

By creating dummy variables for nominal and ordinal, can I use K-means? If not, why? What are the limitations associated with it?

$\endgroup$
2
  • 1
    $\begingroup$ If you have different type of features, consider using Gower similarity coefficient (search this site, to read more). Hierarchical clustering, medoid method and some other allow for using arbitrary distances, including Gower coefficient. And no, you may not use k-means with ordinal or nominal data (again, read on this site about k-means). $\endgroup$
    – ttnphns
    Sep 16 '14 at 8:15
  • 1
    $\begingroup$ You asked the same question on SO before. Why didn't you consider the comments there to improve your question before reposting? This annoys people that could help you, if you ignore their answer... In particular, I pointed out that the question already exists, and you should use search. $\endgroup$ Sep 17 '14 at 8:51
2
$\begingroup$

The clustering algorithm as such usually does not care any more whether your original data were nominal, ordinal or anything else. Instead, most common clustering algorithms only look at distances between data points.

Why? Because clustering attempts to group data points into groups such that all data points within a group are "similar", while data points in different groups are "dissimilar". And similarity can be operationalized via distances. If we have a notion of distance between data points, we can say that two data points are similar if the distance between them is small, and that they are dissimilar if the distance is large. In the end, talking about (dis)similarity and distances amounts to the same thing, but it is more common to discuss clustering in terms of distances.

So it seems like your key question is not so much the clustering algorithm, but defining the distance or (dis)similarity between your data points. Specifically:

  • For ordinal variables, you will need to decide whether, e.g., the distance between A and C is double the one between A and B... or the sum of the distances between A and B and between B and C. Or the distance between A and C could be more than either. Or less.

  • For nominal variables, the simplest approach usually is dummy coding, which translates into a distance of one between instances that differ on the variable. If you have additional structure on your nominals (perhaps some values are "more different" than others), you can include this.

  • Distances between (one-dimensional) ratio variables are usually taken to be the absolute value of the difference, but of course you can do log or power transformations to emphasize small or large absolute differences.

Next, now that you have defined distances between variables, you need to combine them into distances between data points (i.e., collections of variables). Here, you may need to scale stuff to make different dimensions comparable. You can group variables and combine them using the Manhattan, Euclidean, ... distance.

Finally, you toss your distances into k-means or DBSCAN or whatever else.

$\endgroup$
6
  • 1
    $\begingroup$ you toss your distances into k-means. Stephan, what then will be your stance over that many of statements already done on this site, that k-means is appropriate only for Euclidean metric? $\endgroup$
    – ttnphns
    Sep 16 '14 at 8:04
  • $\begingroup$ The clustering algorithm as such usually does not care any more whether your original data were nominal, ordinal or anything else. So, it means that all contemporary clustering methods are totally independent of the notion of the type distance to use. I feel that this statement is historically premature. $\endgroup$
    – ttnphns
    Sep 16 '14 at 8:08
  • $\begingroup$ @ttnphns: re your 1st comment - I don't like or recommend k-means, anyway. It's simple, but that's all it has going for it. Re your 2nd comment - please note I wrote usually. I was going to add a disclaimer that of course there are clustering algorithms that do not simply work on distance matrices, but elected to keep my answer simple. I'd be happy to see an answer that goes into deeper details. $\endgroup$ Sep 16 '14 at 8:19
  • $\begingroup$ Hi Stephan, can you elaborate on the distance part a bit please? I am fairly new to this. $\endgroup$
    – learner
    Sep 16 '14 at 9:35
  • $\begingroup$ I added a paragraph on distances and similarities. Hope it helps. $\endgroup$ Sep 16 '14 at 14:46
1
$\begingroup$

Seriously, use search. This has been answered in detail repeated, so I'm not going to copy and paste it all here.

If you have mixed type data, the first thing to do is understand how to measure similarity. If you don't understand your data, no clustering algorithm will.

k-means only works for linear numerical data. It needs to compute centers (the "means" in "k-means"). If you cannot compute a center that minimizes your distance, k-means will fail to converge. You have to use other algorithms such as hierarchical clustering, PAM, DBSCAN or CLARA, or ... hundreds of others. But first solve the distance problem, they all need a reliable distance measure.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.