Appropriate clustering algorithm with mixed data types

Question

I have a customer data set with several features. These features have mostly different meanings e.g.

• (Currency / Money) Customer monthly spend in $
• (Count) Quantity of service x customer has active with us
• (Count) Quantity of service y that customer has with us
• (Boolean) Is customer a premium services customer
• (Boolean) Is customer some other kind of customer boolean
• (Time) Customer tenure in months
• (Time but entirely different meaning) Average time in a month spent using our call center

I had built a script using simple KMeans only because I've used it before and it's simple to understand.

Then, a post I came across told me to scale my data, that makes sense. But then I did some more research and found that KMeans is not ideal for clustering on binary fields. I have some binary fields, but not all my data are binary. The example features above pretty much cover the nature of the features that I have in my data set.

Can I still use Kmeans for clustering? Are there some more appropriate algorithms to cluster data of this nature?

Edit: On the topic of scaling, I was reading this SO post. From the accepted answer:

... and it is not appropriate to scale a discrete or categorical variable.

Should I scale some of my features and not others? Is this a standard practice? So with the example features above I would scale Customer monthly spend, the two count variables (integers) and the two time features since they are expressed as integers of whole months. Then, I would just leave the booleans, which are just dummied categoricals?

Answer 1

Clustering is an inexact science at the best of times, so there is no definitively correct way of doing this. On the question of scaling your features, considering the following:

You have two features, A and B. A is continuous, and you scale it so that its standard deviation and variance are 1. B is binary/boolean, and you don't scale.

Your cluster centroids along the B axis, will take a value between 0 and 1. In the most extreme case, a centroid could be at exactly 0 or 1. Thus the "most different" any given data point can be from any given centroid is 1, and this would contribute 1 to the Euclidean distance of this point from that cluster.

What about feature A? If a data point is a distance of 1 away from a centroid along the A-axis, it is exactly one variance away. Is this fair? Is a difference of one-sigma comparable with two readings of a binary variable being unequal ? It's hard to say and perhaps problem specific. What is likely true, is that you should scale your continuous data, otherwise, you could find that the variance is 0.001 or 1000 and then if you replayed my argument, it would start to sound silly.

In terms of whether to use KMeans, it's rather unprincipled to do so, and the results can be hard to interpret, but the algorithm itself is simple to understand and performs well, so there's no harm in using it. It's worth conducting sense checks of your results to see if you're getting anything sensible. One check would be to plot distributions of features in each cluster and see whether they separate and generally get less wide. Are the variances of each feature lower within clusters than globally (and much less so than if you took random samples of features with the same size as each cluster) ?

Your data has low enough dimensionality that you might also want to try using (H)DBSCAN as well as k-means and run some of the same sense checks I described.

If you want something more powerful, albeit no more principled, and with the benefit of being visualisable, you could try reducing the dimensionality of your data using t-SNE, UMAP or an Autoencoder and then using (H)DBSCAN. This will have the benefit that your data will go from being discrete along some axes, to looking close to continuous in your lower dimensional space. This will make clustering algorithms like k-means and DBSCAN more suitable. Also, you can plot the data in 2d and see whether the clustering is doing anything sensible.