4
$\begingroup$

How to handle missing values when computing similarity (or distances)? (I have binary feature values and do use the simple matching coefficient, but I feel that the answer to this question may be more general)

I can think of two options:

  • Remove missing values
  • Count missing values as error

But removing null values has the problem that a high score can be achieved with only one/a few values (see example A). And counting missing as error has the problem that real miss matches should be counted higher than missing values (see example B).

Is there a technique that has none of these shortcomings?

Example A

           Instance1 Instance2
  Feature1 missing   missing
  Feature2 1         missing
  Feature3 0         0
  Feature4 missing   1
  Feature5 missing   missing
  Feature6 missing   missing

  Simple-matching-similarity-REMOVE = 1/1 (twice as high as B)
  Simple-matching-similarity-COUNT-AS-ERROR = 1/6

Example B

           Instance1 Instance2
  Feature1 missing   1
  Feature2 1         0
  Feature3 0         0
  Feature4 1         1
  Feature5 1         0
  Feature6 0         missing

  Simple-matching-similarity-REMOVE = 2/4
  Simple-matching-similarity-COUNT-AS-ERROR = 2/6 (twice as high as A)
Improve this question
$\endgroup$
3
  • $\begingroup$ As far as missing is an unknown existing value (which might be really 0 or 1, but you don't know it), the approach that you call "error" is not justified. I think you ought either to completeky remove rows with missing data (= listwise deletion) or to do an imputation (such as, for example, hot-deck imputation). You may also do pairwise deletion, but it is generally considered statistically not a very good choice. $\endgroup$
    – ttnphns
    Commented Dec 10, 2013 at 17:17
  • $\begingroup$ Completely removing rows with missing values is not an option, because in our example, we have 80% missing values, and the features are our target variables (multi-label-classification). We have used imputation for the modelling. But now, I need the similarity measure to evaluate clusters of instances, and this should be done on the raw, unimputated data. $\endgroup$
    – user954923
    Commented Dec 11, 2013 at 10:37
  • 1
    $\begingroup$ 80% of missing data is too much. Imputation is recommended to do with no more than 20% of missing data. $\endgroup$
    – ttnphns
    Commented Dec 11, 2013 at 11:20

2 Answers 2

Reset to default
4
$\begingroup$

As far as I know, there isn't any formal theoretical framework that describes how to do this. A heuristic technique that I've seen used in the past for similar types of missing-data problems is to replace missing values (but for purposes of performing the relative distance calculation only, not for the rest of your analysis!) with a sensibly chosen default value. In your case, a sensible default might be to choose the mean or median of each feature value, tabulated over all instances which do have the feature present. Alternatively, if you wanted to penalize instances with missing values a little bit (i.e., causing them to be treated a little more "conservatively", or more likely dissimilar, since you just don't know for sure) you might substitute missing feature values with the mean value plus some margin, say one sigma or something like that. As I said, it's a heuristic technique, so there's no well-defined "correct answer", you'd have to use your own judgment in deciding precisely which values to substitute for the missing instances.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your answer, I considered your suggestions. I feel, that for my problem, just weighting the score (w/o missing values) with the number of actual matches works quite well.. $\endgroup$
    – user954923
    Commented Dec 11, 2013 at 11:10
1
$\begingroup$

There is a method to calculate Jaccard Similarity with missing values. Please refer this paper.

This method uses matrix factorization technique. To use this method, you first need to calculate the approximate pair-wise Jaccard similarities for your entire dataset. Then you have to use matrix factorization to calculate the exact Jaccard values. However, this method can be used only for small datasets.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.