4
$\begingroup$

I have a question about cluster analysis. Normally when there is less than 10% missing data and its missing at random, then it can be ignored.

But how should I handle the missing data for a cluster analysis if it is higher than 10%? As far as I can see normal statistical methods cannot be used, because then the respondents with missing values will look like each other and then be grouped.

$\endgroup$
3
$\begingroup$

It depends on your similarity measure.

You can define similarity such that it considers missing data to be the same, to be different, to be different with a certain probabilitiy, ... whatever you need. Just don't forget to first work on your data, and distance, then on clustering.

| cite | improve this answer | |
$\endgroup$
3
$\begingroup$

Maybe these are just language issues but I think they are worth clarifying.

Firstly you state missing values with less than 10% missing are ignored. This is not completely true. If less than 10% are missing (and data is scarce and therefore precious), the first approach people employ is to try and impute missing values. If you drop away each input with even one missing value you might end up dropping a lot of the entries. Points are generally only dropped when a lot of their attributes are missing

You state "Normal Statistical Models cannot be used...". This is not completely true. A lot of the imputation techniques end up employing EM techniques with Normal update steps. In a lot of the techniques, missing values are given the same dummy label of (say -1). I would not do this in the case of clustering techniques because it has the possibility of messing up your distance calculations.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.