4
$\begingroup$

I have downloaded the sample codes from kaggle for the randomForest benchmark [URL?] and there's this part that I don't understand.

  appendNAs <- function(dataset, cols) {
  append_these = data.frame( is.na(dataset[, cols] ))
  names(append_these) = paste(names(append_these), "NA", sep = "_")
  dataset = cbind(dataset, append_these)
  dataset[is.na(dataset)] = -1
  return(dataset)
  }

I understand that randomForest does not accept NAs and I usually just impute them with median or rfImpute. Adding extra columns as NAs indicator is new to me. I have looked at the help file for randomForest but didn't anything useful.

Would be great if someone can explain how and why the extra columns work.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
6
$\begingroup$

You somehow have to model the the missingness (NAs). You can either model the variables that contain NAs in order to impute values, or you can include NA indicator values that become a part of your overall model.

When you think about it that way, simply choosing the median of a variable to fill in its NAs is probably not a great model. It's convenient, since it does not involve any of the other variables and it's easy to calculate, but is there any real justification that the particular NA values really would fall close to the median?

Using indicator variables lets you attempt to model the missingness in terms of the other variables.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Will the choice of '-1' affect the prediction? or can I basically put any unrealistic number of my choice? e.g. -999 $\endgroup$
    – King
    Commented May 21, 2012 at 13:44
  • 1
    $\begingroup$ I've heard of replacing NA's with 0 and adding the indicator variable. I believe you want something that's actually near-reasonable, but I don't really know what options are best. Perhaps in the case you mention, they are using 1 for TRUE and -1 for FALSE or something like that. $\endgroup$
    – Wayne
    Commented May 21, 2012 at 16:32
  • 2
    $\begingroup$ I rarely work outside the regression context but I can't imagine why you would replace NAs with 0 instead of mean/median. If want to do better, consider multiple imputation with cross-validation. And always ask yourself why data is missing. $\endgroup$
    – Michael Bishop
    Commented May 21, 2012 at 17:26
  • $\begingroup$ @MichaelBishop: +1 for the "always ask yourself" tip. I thought I'd read replacing with zero in a couple of examples, but can't google up anything, so I guess it was very case-specific. $\endgroup$
    – Wayne
    Commented May 21, 2012 at 19:01
  • $\begingroup$ @MichaelBishop: All the data given are just Quan_#, Cat_#; so there's no way for me to find out why they are missing! $\endgroup$
    – King
    Commented May 22, 2012 at 8:20
7
$\begingroup$

In many cases, the presence of NA is non-random, and can be used as a predictive variable in its own right. For example, knowing whether survey participants chose to disclose a certain fact can be as important as the fact itself. Imputation discards this information, which can reduce predictive performance.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ I like your answer too, but Wayne's similar explanation was 2 minutes faster, so I ticked his. Thank you for helping me understand this step!! $\endgroup$
    – King
    Commented May 21, 2012 at 12:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.