# Clustering a dataset with both discrete and continuous variables

I have a dataset X which has 10 dimensions, 4 of which are discrete values. In fact, those 4 discrete variables are ordinal, i.e. a higher value implies a higher/better semantic.

2 of these discrete variables are categorical in the sense that for each of these variables, the distance e.g. from 11 to 12 is not the same as the distance from 5 to 6. While a higher variable value implies a higher in reality, the scale is not necessarily linear (in fact, it is not really defined).

My question is:

• Is it a good idea to apply a common clustering algorithm (e.g. K-Means and then Gaussian Mixture (GMM)) to this dataset which contains both discrete and continuous variables?

If not:

• Should I remove the discrete variables and focus only on the continuous ones?
• Should I better discretize the continuous ones and use a clustering algorithm for discrete data?
-
You need to find a good distance measure (often the most difficult task in clustering): if you can find a distance measure that correctly and accurately describes how similar (or not) your data items are, then you should not have any problems. –  Andrew May 10 '12 at 10:59
Speaking about those 2 categorical variables you in effect described them as ordinal. Now, what's about the rest 2 "ordinal" variables? How are they different from those? –  ttnphns May 10 '12 at 14:12
They are also discrete, but both of them have a meaningful distance function, i.e. they are interval-based (if I am not messing up the definition of interval-based). –  ptikobj May 10 '12 at 14:43

I've had to deal with this kind of problem in the past, and I think there could be 2 interesting approaches:

• Continuousification: transform symbolic attributes with a sequence of integers. There are several ways to do this, all of which described in this paper. You can try NBF, VDM and MDV algorithms.

• Discretization: transform continuous attributes into symbolic values. Again, many algorithms, and a good lecture on this would be this article. I believe the most commonly used method is Holte's 1R, but the best way to know for sure is to look at the ROC curves against algorithms like EWD, EFD, ID, LD or NDD.

Once you have all your features in the same space, it becomes an usual clustering problem.

Choosing between continuousification or discretization depends on your dataset and what your features look like, so it's a bit hard to say, but I advise you to read the articles I gave you on that topic.

-

K-means obviously doesn't make any sense, as it computes means (which are nonsensical). Same goes for GMM.

You might want to try distance-based clustering algorithms with appropriate distance functions, for example DBSCAN.

The main challenge is to find a distance function!

While you could put a different distance function into k-means, it will still compute the mean which probably doesn't make much sense (and probably messes with a distance function for discrete values).

Anyway, first focus on define what "similar" is. Then cluster using this definition of similar!

-