0
$\begingroup$

I am currently training a number of separate classifiers and I want to use them to create a new Voting classifier.

I currently have the code for the Voting Classifier set up as a separate GridSearch, like so

pipe = Pipeline([
    ('pre_processing', None),
    ('pca', None),
    ('classifier', VotingClassifier(
        voting='soft',
        estimators=[
            ('xgb', xgb.XGBClassifier()),
            ('randomforest', RandomForestClassifier()),
            ('kneighbors', KNeighborsClassifier()),
            ('logit', LogisticRegression()),
            ('svm', SVC()),
        ],
    ),
)
])


voting_model = GridSearchCV(pipe, param_grid=params,
        cv=10, n_jobs=-1, scoring='roc_auc'
).fit(X_train, y_train)

Which is then followed by GridSearch training operations for the individual models like so

pipe = Pipeline([
        ('pre_processing', None),
        ('pca', None),
        ('classifier', LogisticRegression())
    )
    ])

logit_model = GridSearchCV(
    pipe, param_grid=logit_params,
    n_jobs=-1, scoring='roc_auc', cv=10
).fit(X_train, y_train)

I do this since the Voting Classifier does not always perform better than each individual model, so I like to compare the ensemble model to the individual models.

My question is as follows: seeing as the individual models are generated by GridSearch operations as well, will they be the same as the ones generated in the Voting Classifier?

In other words: Would I be able to instead train all of the individual models first and then generate a Voting classifier by including them as parameters? Their hyper parameters already having been set would negate the necessity of running a GridSearch for the VotingClassifier (as I did in the first example) Thus turning the first example in the one below:

vote = VotingClassifier(
    estimators=[
        ('knn', knn),
        ('rf', rf),
        ('xgb', xgb),
        ('logit', logit),
    ],
    voting='soft'
).fit(X_train, y_train)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.