0
$\begingroup$

I was looking at a kernel implementation (for text classification) and the following piece of code got me a little bit confused (I removed part of the features - in order to keep it light - as most of them are similar - e.g. number of negative, positive, neutral words):

pipeline_ = Pipeline([
    ('fu', FeatureUnion([
        ('tfdif_features', Pipeline([
            ('cv', CountVectorizer()),
            ('tfidf', TfidfTransformer()),
            ('tfidf_', Wrapper(RandomForestClassifier())),
        ])),
        ('nb_pos_features', Pipeline([
            ('nb_pos', NumberSelector('posWords') ),
            ('nb_pos_', Wrapper(RandomForestClassifier())),
        ])),
    ])),
    ('xgb', XGBClassifier()),
])

At first I thought that it looks like bagging, as for each feature a base model is created, followed by boosting. But isn't the Random Forest in this case a Decision Tree? Why using Random Forest and not something else (e.g. Logistic regression)?

Any clarification will be greatly appreciated.

$\endgroup$
  • $\begingroup$ Welcome to CV. I think you are asking about the difference between a CART, a logistic regression, a RF, and a sequential learner comprised of two random forests that input into a gradient boosted tree. $\endgroup$ – EngrStudent Jun 20 '19 at 13:02
0
$\begingroup$

But isn't the Random Forest in this case a Decision Tree?

All random forest models are ensembles of decision trees. Random forest constructs many decision trees using a randomization procedure.

Why using Random Forest and not something else (e.g. Logistic regression)?

Because the author of this code made a deliberate choice. You'll have to ask the author why they chose Random Forest instead of a different method. Generically, tree induction methods like random forest are very flexible, whereas logistic regression is merely a linear model.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for your answer! $\endgroup$ – moz_szt Jun 21 '19 at 8:14
  • $\begingroup$ @moz_szt if you’ve found my answer helpful, please consider upvoting and/or accepting it. More information about how the site works can be found in the help center. $\endgroup$ – Sycorax Jun 21 '19 at 13:25
0
$\begingroup$

What the model is intending to do is use the output of RandomForest1 and RandomForest2 as features into the XGBoost Classifier

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for your answer. I understood that part, but I didn't quite understood why using a Random Forest for only one feature? Why not using Logistic Regression or something else? $\endgroup$ – moz_szt Jun 19 '19 at 13:59
  • $\begingroup$ @moz_szt, the TFIDF vectorizer will translate each row to multiple features. It is not a single feature. $\endgroup$ – Anant Gupta Jun 19 '19 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.