3
$\begingroup$

I am tasked with performing a clustering exercise for a consumer survey dataset with the hopes of finding distinct consumer segments.

In the past, I've done it using a variety of techniques- hierarchical methods, EM etc. but the dataset has been much smaller with perhaps 12-15 variables.

I've used dimensionality reduction as a starting point and that has helped with smaller number of variables but with over a 100 variables, I'm a little befuddled. The dataset includes mostly numerical but also some categorical data.

How would I go about such an exercise? Distance measures in higher dimensions are tricky and so I'm seeking some guidance here.

A word about the tools of choice- I would like to run it in R but it'll most likely murder my laptop. Any specific database you guys could recommend?

Improve this question
$\endgroup$
2
  • $\begingroup$ A reference I found that might be helpful: www-users.cs.umn.edu/~kumar/papers/high_dim_clustering_19.pdf $\endgroup$
    – shadowtalker
    Jan 25, 2015 at 20:52
  • $\begingroup$ (1) Have you tried using the Gower distance? (2) The first thing I'd probably do is to go through the 100 variables and see whether some meaningful indexes can be computed from them that summarise their "message" in much lower dimensions. $\endgroup$
    – Christian Hennig
    Sep 12, 2021 at 13:35

1 Answer 1

Reset to default
0
$\begingroup$

You could try the LowRankModels package in julia. It's something of a generalized PCA approach. It supports boolean and cardinal data types as well as real numbers, so it has that going for it as well.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks man! Any other packages you know of that might be helpful? $\endgroup$
    – Nepze Tyson
    Jan 22, 2015 at 17:57
  • $\begingroup$ That's the only one I've seen with boolean and cardinal data support. I'm sure that R has some support for PCA type analysis, but I'm not sure how to handle the categorical data with standard PCA. $\endgroup$
    – user1348
    Jan 22, 2015 at 19:32
  • 1
    $\begingroup$ Obviously if one wanted to go that way, one could use PCA or something similar for the continuous variables and summarise the categorical ones in other ways (e.g., Multiple Correspondence Analysis). $\endgroup$
    – Christian Hennig
    Sep 12, 2021 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.