0
$\begingroup$

I am trying to create an ensemble model. I tried some classical models, in which RandForest's accuracy is around 85% and other models like Decision trees, NNet has accuracy from 35%-63%. So I need some models which are close to the working of Randforest or its variants. I am using R and my model is a Regression Model

$\endgroup$
  • $\begingroup$ Check out RWeka package... it has all the decision tree based classifiers which are similar to Random Forest. Also if your Nnet is performing poorly it must be because of poor training or poor architecture. NNet output should be same if not better than Random Forest, as neural network can theoretically simulate any function given sufficient training and network size. $\endgroup$ – Gaurav Jul 8 '16 at 6:43
  • $\begingroup$ Try the xgboost package. It has gradient boosted trees, which is an ensemble model and a generalization of adaboost, using trees as the weak classifier. In addition, there is the generalized boosted regression model package, gbm, and the original adaboost is in the ada package. The advantage xgboost has over gbm is speed. Xgboost also support using linear models as the weak classifier. $\endgroup$ – aichao Jul 8 '16 at 13:04
0
$\begingroup$

I would suggest to go with the "Extreme Gradient Boosting" which is based on Gradient Boosting method, but instead of using only linear model solver, it includes tree based algorithms too. It has become go-to method in Kaggle competitions for regression analysis surpassing Random Forests. Check out Anthony Goldbloom's talk about "how to win Kaggle competitions".

If you are using R, go with "xgboost" package which implements Extreme Gradient Boosting algorithm. I was able to to reduce RMSE of my prediction by 10 times compared to that of RF.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.