I am working on a project by using a high dimensional data set. Close to 50000 Obs. with 392 Variable.

I used lasso to reduce it to this point from a total of 1200 variables. And the whole data set is grouped into several clusters. The data is of a product and its specifications and other relevant details as covariates and time to get sold is the response.

I am trying to predict the time with the help of covariates and I am using randomForestSRC package.

    options(rf.cores = 8)
    rfdata <- data.frame(time,factors,agewitinmy,exclusivity,specs,financial)
    obj <- rfsrc(time ~ ., data = rfdata[(1:500), ],nsplit = 10)
    pre <- predict(obj,rfdata[(500:550), ])
    results <- cbind(data.frame(time[500:550],pre$predicted))

But the accuracy is not good, am I doing something wrong here ?​ Can I adjust the accuracy by doing some changes. Or is it because of the high number of variables ?

Thanks for your time and response.

  • $\begingroup$ I'm not sure that the lasso procedure is useful here because the random forest model is implicitly interactive, while the lasso only looks at linear effects in the equation that you specify. So a feature that's not important on its own (in lasso) might be important in the presence of another (in random forest). $\endgroup$
    – Sycorax
    Apr 29 '15 at 16:49

Pre-selecting with lasso may not be beneficial; can check both scenarios. RF was designed for thousands of predictors, so this should not in itself be an issue. Just need to make sure the number of trees is appropriate for the number of predictors (check error stabilization).

Several general possibilities for improvement include: removing worst predictors; removing correlated predictors; adjusting for predictors with a very wide range of classes; weighted RF if there is class imbalance (although improvement they report is rather small).

  • $\begingroup$ Thank you, I checked with out using lasso and it is giving better prediction. I think lasso removed/made the coefficient of some variables to zero which was important for prediction in RF. $\endgroup$
    – Sam
    May 2 '15 at 12:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.