4
$\begingroup$

I know some strategies of imputing the missing data, for example, using filling with zeros, using mean, median or the most frequent values.

So what I don't quite understand till this point-how can the missing values be predicted in Python using some machine learning techniques such as RandomForestRegressor?

What steps should be taken to imputing the values by predicting them with RandomForest (or maybe other models, such knn, for example).

Improve this question
$\endgroup$
5
  • $\begingroup$ You can use variable in which you have missing values as dependent variable and all the rest as independent variables and build a predictive model. If you have more than one variable with missing values you may use a separate model for each one or develop multidimensional model (e.g. KNN). $\endgroup$
    – Łukasz Deryło
    Commented Jul 13, 2017 at 9:58
  • $\begingroup$ @ŁukaszDeryło see my comment to the answer below, IMO single imputation is not quite the answer (multiple imputation is [better] able to account for the uncertainty which occurs when replacing missing values). $\endgroup$
    – IWS
    Commented Jul 13, 2017 at 10:58
  • 1
    $\begingroup$ @IWS I agree that MI is better than SI, but i think HalfPintBoy asked how it is even possible to use learning techniques (RF, KNN, ... ) to impute missing values, not what is the best way of doing this. $\endgroup$
    – Łukasz Deryło
    Commented Jul 13, 2017 at 11:13
  • $\begingroup$ @ŁukaszDeryło Fair enough, I just wanted to make clear that imputation of missing values using these models only once is a tricky way to go, which often results in incorrectly 'precise' estimates/standard errors. What we seem to agree on is IMHO most important: if performed correctly, multiple imputation can cope with this problem. $\endgroup$
    – IWS
    Commented Jul 13, 2017 at 11:39
  • $\begingroup$ the "mice" package in "R" has "rf" as a method for variable imputation. You might look there. I hear there are libraries for wrapping r and running it in python. ;) $\endgroup$
    – EngrStudent
    Commented Jul 14, 2017 at 14:13

2 Answers 2

Reset to default
1
$\begingroup$

What I would do:

For each variable in your data I would regress it with the rest of the data, so for variable v1, you should regress it with v2 ... vn, that do not have an overlap in missing data with v1. You could save the names or indexes of the subjects that have missing data for v1 to a list and determine the overlap of missing values between variable one and the other variables. You should then only use the variables that do not have overlapping missing data with v1. After adding such an if statement, v2 would be regressed with v1, v3 ... vn, and so on. This way you will have a regression based on non-missing data.

After fitting the regression you can use the predictors (v2 ... vn) to predict the missing data in v1. Because you already know which subjects have missing data for v1, you can use the data of these subjects for the other variables: v2 ... vn to predict the missing data in v1 and then impute it.

By doing this for each variable, you will get an imputed dataset.

If you are not yet already, you can use Pandas to easily index the variables and subjects that have missing data.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Note that this only works if the missings only occur in one variable. Even then, this 'single imputation' is based on the assumption that the imputation model final estimates are unbiased and completely accurate. That is why multiple imputation has been proposed. By replacing not with one value, but with multiple ones, and introducing some randomness to the imputation models, the variation across the replacement values/subsequent analyses becomes a measure for the certainty for the replacement value. $\endgroup$
    – IWS
    Commented Jul 13, 2017 at 10:54
  • $\begingroup$ See the references I've provided in the answer to this question for background information: stats.stackexchange.com/questions/257672/… $\endgroup$
    – IWS
    Commented Jul 13, 2017 at 10:54
  • $\begingroup$ @IWS He specifically asked: "What steps should be taken to imputing the values by predicting them with RandomForest (or maybe other models, such knn, for example)." so even though I completely agree with you, I think my answer does describe the steps you can take to impute a dataset by using RandomForestRegression $\endgroup$
    – Eloyg
    Commented Jul 13, 2017 at 11:21
  • 1
    $\begingroup$ Still I do not see your answer working: Could you add how to handle the situation where there are missing values in more than one variable? i.e. how are you going to find a replacement value for variable 1 of individual 1, when variables 2, 6 and 9 are also missing for this individual? If regular regression was used, you'd not be able to complete the regression formula if these variables are the predictors (or do you assume them normal/0?). Moreover, individual 2 could be missing values in different variables. So the question is if there are enough left to build your initial regression models. $\endgroup$
    – IWS
    Commented Jul 13, 2017 at 11:26
  • $\begingroup$ @IWS You could add an extra if statement, to say: if predictors do not miss any values for the subjects that have missing data for 1; if none of the missing data intersects, the predictors can be used for regression $\endgroup$
    – Eloyg
    Commented Jul 13, 2017 at 11:28
0
$\begingroup$

If you do not need to impute the missing values at all you could also try one of the following strategies:

  1. using ternary decision trees: the nodes propagate the sample with a missing value to a third branch,
  2. propagate the samples with a missing value in both child nodes,
  3. randomly propagate the samples with a missing value in one of the child node.

These approaches allow to build the model even if there are missing values and are not expensive from the computational point of view.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.