For my classification, I use several algorithms available in WEKA, but with limited number of features. I got some accuracy levels with the algorithms I used and I tried improving the accuracies using Ensemble methods. I used Boosting and Bagging for that. But the outcome I got is strange.
Following figures show, the accuracies I got with and without bagging and boosting.
What I expected to see after bagging and boosting was some improvements of the accuracy levels. But you can see that,
- when I used Random Forest algorithm with boosting, the accuracy has been decreased. I can understand that why accuracies do not get improved. But I do not know why they get decreased.
- In bagging, both Random forest and Naive Bayed have shown unexpected behaviors.
I did all these experiments in WEKA. I need to know why this happens? And also do they happen since I have a limited amount of features for my classification?