0
$\begingroup$

I was looking at a kernel implementation (for text classification) and the following piece of code got me a little bit confused (I removed part of the features - in order to keep it light - as most of them are similar - e.g. number of negative, positive, neutral words):

pipeline_ = Pipeline([
    ('fu', FeatureUnion([
        ('tfdif_features', Pipeline([
            ('cv', CountVectorizer()),
            ('tfidf', TfidfTransformer()),
            ('tfidf_', Wrapper(RandomForestClassifier())),
        ])),
        ('nb_pos_features', Pipeline([
            ('nb_pos', NumberSelector('posWords') ),
            ('nb_pos_', Wrapper(RandomForestClassifier())),
        ])),
    ])),
    ('xgb', XGBClassifier()),
])

At first I thought that it looks like bagging, as for each feature a base model is created, followed by boosting. But isn't the Random Forest in this case a Decision Tree? Why using Random Forest and not something else (e.g. Logistic regression)?

Any clarification will be greatly appreciated.

$\endgroup$
1
  • $\begingroup$ Welcome to CV. I think you are asking about the difference between a CART, a logistic regression, a RF, and a sequential learner comprised of two random forests that input into a gradient boosted tree. $\endgroup$ Jun 20, 2019 at 13:02

2 Answers 2

0
$\begingroup$

But isn't the Random Forest in this case a Decision Tree?

All random forest models are ensembles of decision trees. Random forest constructs many decision trees using a randomization procedure.

Why using Random Forest and not something else (e.g. Logistic regression)?

Because the author of this code made a deliberate choice. You'll have to ask the author why they chose Random Forest instead of a different method. Generically, tree induction methods like random forest are very flexible, whereas logistic regression is merely a linear model.

$\endgroup$
2
  • $\begingroup$ Thank you for your answer! $\endgroup$
    – moz_szt
    Jun 21, 2019 at 8:14
  • $\begingroup$ @moz_szt if you’ve found my answer helpful, please consider upvoting and/or accepting it. More information about how the site works can be found in the help center. $\endgroup$
    – Sycorax
    Jun 21, 2019 at 13:25
0
$\begingroup$

What the model is intending to do is use the output of RandomForest1 and RandomForest2 as features into the XGBoost Classifier

$\endgroup$
2
  • $\begingroup$ Thank you for your answer. I understood that part, but I didn't quite understood why using a Random Forest for only one feature? Why not using Logistic Regression or something else? $\endgroup$
    – moz_szt
    Jun 19, 2019 at 13:59
  • $\begingroup$ @moz_szt, the TFIDF vectorizer will translate each row to multiple features. It is not a single feature. $\endgroup$ Jun 19, 2019 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.