0
$\begingroup$

I was looking at a kernel implementation (for text classification) and the following piece of code got me a little bit confused (I removed part of the features - in order to keep it light - as most of them are similar - e.g. number of negative, positive, neutral words):

pipeline_ = Pipeline([
    ('fu', FeatureUnion([
        ('tfdif_features', Pipeline([
            ('cv', CountVectorizer()),
            ('tfidf', TfidfTransformer()),
            ('tfidf_', Wrapper(RandomForestClassifier())),
        ])),
        ('nb_pos_features', Pipeline([
            ('nb_pos', NumberSelector('posWords') ),
            ('nb_pos_', Wrapper(RandomForestClassifier())),
        ])),
    ])),
    ('xgb', XGBClassifier()),
])

At first I thought that it looks like bagging, as for each feature a base model is created, followed by boosting. But isn't the Random Forest in this case a Decision Tree? Why using Random Forest and not something else (e.g. Logistic regression)?

Any clarification will be greatly appreciated.

Improve this question
$\endgroup$
1
  • $\begingroup$ Welcome to CV. I think you are asking about the difference between a CART, a logistic regression, a RF, and a sequential learner comprised of two random forests that input into a gradient boosted tree. $\endgroup$
    – EngrStudent
    Commented Jun 20, 2019 at 13:02

2 Answers 2

Reset to default
0
$\begingroup$

But isn't the Random Forest in this case a Decision Tree?

All random forest models are ensembles of decision trees. Random forest constructs many decision trees using a randomization procedure.

Why using Random Forest and not something else (e.g. Logistic regression)?

Because the author of this code made a deliberate choice. You'll have to ask the author why they chose Random Forest instead of a different method. Generically, tree induction methods like random forest are very flexible, whereas logistic regression is merely a linear model.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for your answer! $\endgroup$
    – moz_szt
    Commented Jun 21, 2019 at 8:14
  • $\begingroup$ @moz_szt if you’ve found my answer helpful, please consider upvoting and/or accepting it. More information about how the site works can be found in the help center. $\endgroup$
    – Sycorax ♦
    Commented Jun 21, 2019 at 13:25
0
$\begingroup$

What the model is intending to do is use the output of RandomForest1 and RandomForest2 as features into the XGBoost Classifier

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for your answer. I understood that part, but I didn't quite understood why using a Random Forest for only one feature? Why not using Logistic Regression or something else? $\endgroup$
    – moz_szt
    Commented Jun 19, 2019 at 13:59
  • $\begingroup$ @moz_szt, the TFIDF vectorizer will translate each row to multiple features. It is not a single feature. $\endgroup$
    – Anant Gupta
    Commented Jun 19, 2019 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.