# Clustering with numerical variables and one non numerical

I'd like some help with an issue that might seem easy but I'm stuck in my analysis.

I'm working with R on a dataframe containing 15 variables: 14 numerical or Integer and 1 factor. I'd like to do some clustering on my dataframe but here is the issue:

Usually, when I'm working with factor variables for clustering, I do a MCA analysis and keep the coordinates as numerical variables on which I do my clustering (usually kmeans).

The thing is that I don't really have any idea on how I can work with this only factor factor variable.

Should I use dummies? Any other idea?

Yes, you can use dummy coding.

Imagine the factor (categorical) variable $X$ has $N$ possible values: 1,2,3,, N. Then transform it into a vector of $N$ variables where all of them are 0 except the coordinate corresponding to the value of $X$. Example $X=2$ turns into $(0;1,0,....,0)$. You can read this.

Then concatenate this vector to the other variables and do the clustering on the concatenated vectors. I don't know if your kmeans implementation does it all for you, but it might be useful to normalize the data before clustering, especially when mixing dummy coding with other variables.

Another idea that is essentially equivalent is to define your own distance. By default, the kmeans will usually use the euclidean distance between vectors $x$ and $y$ of variables:

$$d^2(x,y)=\sum_{i=1}^n(x_i-y_i)^2$$

For a categorical variable (say it is the one at the first coordinate), $(x_1-y_1)^2$ does not mean much since categories $x_1=1$ and $y_1=7$ would be considered to be farther than categories $x_1=1$ and $y_1=2$. Instead we use a special binary distance that is just 0 when $x_1=y_1$ and 1 otherwise. It is denoted $1_{x_1\neq y_1}$. Then we use this new distance:

$$d'^2(x,y)=\lambda1_{x_1\neq y_1}+\sum_{i=2}^n(x_i-y_i)^2$$

This idea with $\lambda=2$ is strictly equivalent to dummy coding. This helps understanding how dummy coding sees the data in terms of distance: the contribution of the categorical variable to the distance between two points is:

• +2 if different categories
• +0 if same categories

Note: when you rescale your data, you actually change $2$ into whatever you want.

• Thank you for your answer. I will scale my dataframe using the scale function in R. But I didn't understand your second idea. Could you explain it a bit more or use an example ? Dec 12, 2017 at 12:33
• I've edited the answer. It's not so much important to understand this other idea indepth since it does the same as dummy coding. Dec 12, 2017 at 13:14

While dummy coding does allow to use kmeans, it just adds to the scaling problems of kmeans, which is really sensitive to variable scaling. It is a "hack", not a solution.

KMeans, from a theoretical point of view, only makes sense on continuous variables of the same units.

So I suggest that rather than "hacking" the data to be accepted by the algorithm, and then eventually wondering what the result actually means by reverse engineering what you did, I suggest that you first try to formalize your problem: what would be a good, meaningful cluster in your case? Put this into an equation that gives you a quality (you will eventually need this to evaluate any result, including the kmeans results above), but it must really be relevant to your problem. Gun you can try to find a) some existing algorithm that may be able to find a good solution, b) try various algorithms and evaluate them wrt. your task, c) devise your own algorithm/variant that is better at finding a good solution. And of course you may need to return to your mathematical definition of "quality" several times, because you will often find that it does not yet do what you want...