0
$\begingroup$

I am new to Data Science and have been studying the methods of stacking to find out if it can meet the following fact, but I did not find or understand evidence that it can or cannot work.

Let's imagine a dataset divided into two folds (train and test). For the Layer 1 models, the only data I can get is the predictions from the test fold. Would stacking principles work using those predictions as a feature, or could the fact that each Layer 1 model is using and predicting the same fold observations cause problems like overfitting?

I ask this question because I have seen that almost every time people talk about stacking, mechanisms like k-fold cross-validation are applied, but I can't get the predictions for those mechanisms, only the ones made for the test set.

Improve this question
$\endgroup$
3
  • $\begingroup$ Cross-posted at datascience.stackexchange.com/q/68464/55122 $\endgroup$
    – Ben Reiniger
    Feb 21, 2020 at 15:59
  • $\begingroup$ I don't completely understand your question. The second layer model gets trained on predictions from the base models, and these need to be outside of the base models' training data (else the meta-model will tend to clone the most-overfit base model). This can be done with a dataset completely separate from the base training set, but more commonly one makes k-fold test-fold predictions to keep the dataset sizes as large as possible. $\endgroup$
    – Ben Reiniger
    Feb 21, 2020 at 16:02
  • $\begingroup$ Hello @BenReiniger, thanks for your answer. So, if I understand correctly, techniques used for training level-1 models doesn't affect the stacking process (even though it can affect to the stacking results quality) as long as those model predictions (used in the next step to train the meta model) are made over the same data? Avoiding if the fact that better Layer-1 models imply a better result of the stacking, my question was more directed to the stacking underlying process, which I didn't know if it would be affected by doing that kind of data splitting $\endgroup$
    – dg1996
    Feb 21, 2020 at 19:28

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.