# similarity measures with missing values

How to handle missing values when computing similarity (or distances)? (I have binary feature values and do use the simple matching coefficient, but I feel that the answer to this question may be more general)

I can think of two options:

• Remove missing values
• Count missing values as error

But removing null values has the problem that a high score can be achieved with only one/a few values (see example A). And counting missing as error has the problem that real miss matches should be counted higher than missing values (see example B).

Is there a technique that has none of these shortcomings?

Example A

           Instance1 Instance2
Feature1 missing   missing
Feature2 1         missing
Feature3 0         0
Feature4 missing   1
Feature5 missing   missing
Feature6 missing   missing

Simple-matching-similarity-REMOVE = 1/1 (twice as high as B)
Simple-matching-similarity-COUNT-AS-ERROR = 1/6


Example B

           Instance1 Instance2
Feature1 missing   1
Feature2 1         0
Feature3 0         0
Feature4 1         1
Feature5 1         0
Feature6 0         missing

Simple-matching-similarity-REMOVE = 2/4
Simple-matching-similarity-COUNT-AS-ERROR = 2/6 (twice as high as A)

• As far as missing is an unknown existing value (which might be really 0 or 1, but you don't know it), the approach that you call "error" is not justified. I think you ought either to completeky remove rows with missing data (= listwise deletion) or to do an imputation (such as, for example, hot-deck imputation). You may also do pairwise deletion, but it is generally considered statistically not a very good choice. – ttnphns Dec 10 '13 at 17:17
• Completely removing rows with missing values is not an option, because in our example, we have 80% missing values, and the features are our target variables (multi-label-classification). We have used imputation for the modelling. But now, I need the similarity measure to evaluate clusters of instances, and this should be done on the raw, unimputated data. – user954923 Dec 11 '13 at 10:37
• 80% of missing data is too much. Imputation is recommended to do with no more than 20% of missing data. – ttnphns Dec 11 '13 at 11:20