2
$\begingroup$

I am working on a multiclass classification problem which uses sentiments extracted from review texts as feature space and tries to predict the numerical ratings. I am interested in using bagging approach to make predictions. My question is about the appropriateness of using Logistic Regression as the base learner in bagging. The reason why I am interested in Logistic Regression is because I came to know that TF-IDF and Logistic Regression is a powerful combo.

$\endgroup$
2
$\begingroup$

You definitely can. You can use bagging with any type of classifier. However, because bagging is an ensemble method, and logistic regression is a stable classifier, they are not a powerful combo. On the other hand, decision trees are unstable classifiers and they work well when combined in ensembles. The reason for this is that in order for your ensemble to work well, the classifiers in the ensemble should produce independent mistakes. Unstable classifiers, such as decision trees, tend to perform significantly different when the training set is slightly perturbed. This is why the ensemble methods use bootstrap sampling to create a separate training set for all classifiers in the ensemble.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.