I am currently training a number of separate classifiers and I want to use them to create a new Voting classifier.
I currently have the code for the Voting Classifier set up as a separate GridSearch, like so
pipe = Pipeline([ ('pre_processing', None), ('pca', None), ('classifier', VotingClassifier( voting='soft', estimators=[ ('xgb', xgb.XGBClassifier()), ('randomforest', RandomForestClassifier()), ('kneighbors', KNeighborsClassifier()), ('logit', LogisticRegression()), ('svm', SVC()), ], ), ) ]) voting_model = GridSearchCV(pipe, param_grid=params, cv=10, n_jobs=-1, scoring='roc_auc' ).fit(X_train, y_train)
Which is then followed by GridSearch training operations for the individual models like so
pipe = Pipeline([ ('pre_processing', None), ('pca', None), ('classifier', LogisticRegression()) ) ]) logit_model = GridSearchCV( pipe, param_grid=logit_params, n_jobs=-1, scoring='roc_auc', cv=10 ).fit(X_train, y_train)
I do this since the Voting Classifier does not always perform better than each individual model, so I like to compare the ensemble model to the individual models.
My question is as follows: seeing as the individual models are generated by GridSearch operations as well, will they be the same as the ones generated in the Voting Classifier?
In other words: Would I be able to instead train all of the individual models first and then generate a Voting classifier by including them as parameters? Their hyper parameters already having been set would negate the necessity of running a GridSearch for the VotingClassifier (as I did in the first example) Thus turning the first example in the one below:
vote = VotingClassifier( estimators=[ ('knn', knn), ('rf', rf), ('xgb', xgb), ('logit', logit), ], voting='soft' ).fit(X_train, y_train)