I am using
sklearn.ensemble however i am puzzled with the results.
Consider following algorithms:
alg1 = LogisticRegression() alg2 = GradientBoostingClassifier() alg3 = GaussianNB()
Then when I ensemble and fit the models with following weights:
eAlg = VotingClassifier(estimators=[('LR',alg1),('GB',alg2),('NB',alg3)], voting='hard', weights=[1,5,1]) eAlg.fit(X_train,y_train) pred_ens = eAlg.predict(X_test)
I get following result:
np.mean(pred_ens == y_test) #0.91
(I am predicting only 0/1 classes)
Just for my curiosity, I put
weight=5 to GB. Doesn't it mean that even if other two classifiers says otherwise, the model will follow GB? So it should be equal to GB alone.
Because if i train only GB alone, i get slightly higher score. How is that possible?