# Is this kind of stacked ensemble method prone to over-fitting?

I am working on a stacked ensemble method. I trained three classifiers as my first-layer models and one Logistic Regression as my second layer model. I then stacked both the first-layer models and second-layer model and trained it as a Stacked Classifier. Now, I get a perfect score of 1.0 when I measure performance on my test set. This makes me suspicious.

Can you tell me, are these kind of models prone to over-fitting?

My code looks something like this:

# Creating the first-layer models
clf_knn =KNeighborsClassifier(n_neighbors= 5, algorithm= 'ball_tree')
clf_decision_t = DecisionTreeClassifier(min_samples_leaf = 5, min_samples_split =
15, random_state=500)
clf_naive_b = GaussianNB()

# Creating second-layer model (meta-model)
clf_logistic_r = LogisticRegression()

# Creating and fitting the stacked model
clf_stack = StackingClassifier(classifiers=[clf_knn, clf_decision_t , clf_naive_b ],
meta_classifier=clf_logistic_r )
clf_stack.fit(X_train, y_train)

# Evaluate the stacked model’s performance
print("Accuracy: {:0.4f}".format(accuracy_score(y_test,
clf_stack.predict(X_test))))


I used scikit and mlxtend libraries

• Could you please tell us more about the data, the problem and your validation methodology? Have you rerun those tests? Are you able to perform repeated k-fold for at least 50 cycles of training and testing? – Piotr Rarus - Reinstate Monica Dec 13 '19 at 11:27
• Thank you for your answer. Indeed I should use cross-validation. Yohann suggested to use StackingCVClassifier , which is a newer version of StackingClassifier. It has cross-validation integrated. – Horbaje Dec 13 '19 at 13:19