Cluster categorical data (survey data)

Question

I have a dataset containing around 800 observations:

It's a dataset collected via a survey; each row is a dataset filled with information re. diet habits, physical activity, the fact of taking supplements or not, personal goals, type of medication (if any) taken and so on.

Majority of features are categorical (aka factors), except one - item_total - which is the amount each user spent on some product.

x My objective is to cluster these questionnaires based on the available features. I use the R language for performing this task.

My current approach is: 1. I calculate gower distance via daisy package aka dissimilarity matrix

gower.dist <- daisy(df[,-1], metric = c("gower"))

where df is the dataframe.

2) I use agglomerative hierarchical clustering.

aggl.clust.c <- hclust(gower.dist, method = "complete")

3) I find the optimal amount of clusters via the silhouette method.

I would like to check other clustering techniques out. Keep in mind that the features are factors so k-mean -ish techniques woulds be underperforming if not impossible at all to apply.

Any suggestions/sources I would need to check out? Thanks in advance! V

Answer 1

Assuming a distance measure that's valid for discrete data (e.g. the Gower distance you're using), any clustering algorithm based entirely on pairwise distances would be a valid option. The thing you want to avoid is methods that assume a continuous input space (e.g. by computing centroids/means of data points as in k-means, or defining continuous densities as in Gaussian mixture models).

You could explore different linkage methods for hierarchical clustering (e.g. single, average, and complete linkage; Ward linkage isn't appropriate since it requires computing centroids). Some other popular distance-based clustering algorithms include k-medoids, DBSCAN, and OPTICS. Spectral clustering is another possibility, as long as you compute the affinity matrix in a way that respects the discrete data.

Answer 2

Careful using similarity measures on categorical data. Gower’s distance (and the like) calculate distance between categorical variables using a “same not same” measure. That is, if your variable were country and one was Canada and the other USA it would be a 0 vs a 1 if they were both Canada. This is an issue because this is very limiting. USA and Canada may be different but they may be less different than say USA and Japan. Compounding this effect over all of your variables could lead to some not so meaningful measures of similarity.

Answer 3

You can try to "brute force" find a working solution by experimenting with different distances such as Gower's distance (along with hclust, for example), and some clustering algorithms such as k-modes for categoricial data...

But at some point you'll likely wonder whether your result is good. And then you're back to the drawing board, and you'll have to decide on paper what properties a good clustering is supposed to have... The better approach is to first think about what you need - what would be a good solution, which will be very much Applikation depdendent - and then choose an algorithm that can optimize the identified quality properties.