I'm learning using R to build predictive models recently by myself and have many questions on how to attack a question. I'm given a data set of 8000 observations with 300 features. My goal is to build a predictive model to predict a target column of continuous values. I'm not given where the data is coming from, nor the actual meaning of the features (they are listed just as f1 to f300). 5 features are categorical. Almost all features have some missing values. Only 20 observations are complete cases. I have several questions:

  1. How to deal with the missing data? Would either MICE or Amelia package in r for multiple imputation be a good choice, but then how to deal categorical missing data?

  2. Should I do data imputation before building a predictive model or the other way around?

  3. Would random forest be a good choice in this case? How many features should be included in the decision tree?

Thanks in advance for any suggestions.

  • $\begingroup$ A problem where you do not know the meaning of the features is not a realistic scenero to use to learn predictive modeling. $\endgroup$ Commented Oct 11, 2017 at 23:36
  • 1
    $\begingroup$ @MatthewDrury, that's what all these data scientists do now $\endgroup$
    – Aksakal
    Commented Feb 20, 2018 at 15:55
  • $\begingroup$ As a long time data scientista now intamately involved in training the next generation of data scientist, I can assure you that is not a universal truth. $\endgroup$ Commented Feb 20, 2018 at 15:57

1 Answer 1

  1. There are several possibilities for handling missing data. A typical easy one is imputing the median for continuous and the modus for discrete predictors. Other more sophisticated methods are available (like e.g. imputation with random forest, see here for some possibilites with R-package mlr: http://mlr-org.github.io/mlr-tutorial/devel/html/impute/index.html)

  2. As most of the algorithms for predictive modeling cannot handle missing data, you should do imputation before building a model

  3. Random Forest (randomForest or ranger in R) and linear model (lm in R) are good first options for regression problems. Better results with a bit of parameter tuning can usually be obtained by boosting methods (e.g. with the xgboost package), but a bit more experience is needed for this.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.