How to handle missing values when computing similarity (or distances)? (I have binary feature values and do use the simple matching coefficient, but I feel that the answer to this question may be more general)
I can think of two options:
- Remove missing values
- Count missing values as error
But removing null values has the problem that a high score can be achieved with only one/a few values (see example A). And counting missing as error has the problem that real miss matches should be counted higher than missing values (see example B).
Is there a technique that has none of these shortcomings?
Instance1 Instance2 Feature1 missing missing Feature2 1 missing Feature3 0 0 Feature4 missing 1 Feature5 missing missing Feature6 missing missing Simple-matching-similarity-REMOVE = 1/1 (twice as high as B) Simple-matching-similarity-COUNT-AS-ERROR = 1/6
Instance1 Instance2 Feature1 missing 1 Feature2 1 0 Feature3 0 0 Feature4 1 1 Feature5 1 0 Feature6 0 missing Simple-matching-similarity-REMOVE = 2/4 Simple-matching-similarity-COUNT-AS-ERROR = 2/6 (twice as high as A)