# Why do we generate out-of-fold predictions for meta-ensembling/stacking?

Here's the guide I'm looking at: http://blog.kaggle.com/2016/12/27/a-kagglers-guide-to-model-stacking-in-practice/

Here's the relevant excerpt: The main point to take home is that we’re using the predictions of the base models as features (i.e. meta features) for the stacked model. So, the stacked model is able to discern where each model performs well and where each model performs poorly. It’s also important to note that the meta features in row $$i$$ of train_meta are not dependent on the target value in row $$i$$ because they were produced using information that excluded the target_i in the base models’ fitting procedure.

Could somebody elaborate on why it is important that the meta features are not dependent on the corresponding label?

## 3 Answers

Different machine learning algorithms can have different degrees of overfitting on the training data. If you use meta features dependent on the target value, you will always favor the algorithm that is more overfitted. To avoid this favor, you should use data not dependent on the target value, i.e. cross-validation.

What the text mean is that the metafeatures are independent of test, since they were learned with train data, that is important because otherwise you would have information leak, and you will fooling yourself believing tour test score is good

Imagine one of the models is a Random Forest. They usually have 100% accuracy if you use the predictions on the training set. Another model, on top of this, would use the predictions of the Random Forest believing that "Random Forest prediction is always right", since they are always equal to the label.

So it would just have the same predictions that the Random Forest, and therefore, the same accuracy.