# how do you stack models with 100% training accuracy?

Suppose I have several models, one of them $M$ has a 100% training accuracy.

So regardless of how a stack the models, the stacked model is just M. e.g. If I use a linear model to stack them, then the new model is just $\hat{y}_\text{stack} = 0\cdot 1 + 0 \cdot \hat{y}_1 + 0 \cdot \hat{y}_2 + \dots + 1\cdot \hat{y}_M$

And I doubt $M$ is overfitting, because regardless of how I tune $M$, the test accuracy is only maximal when the training accuracy is 100%. In other words, I have not encountered a situation where training accuracy is increasing while the test accuracy is decreasing.

Is it impossible to stack models in this situation?

edit: None of the models has a test accuracy of 100% (but $M$ scores the highest), hence the need for stacking.

• If you have already a model with 100% accuracy what's the advantage you want to obtain with stacking?
– GGA
Jan 27, 2017 at 7:43
• @GGA To improve test accuracy Jan 27, 2017 at 7:46
• Wouldn't be the same to use only the model with 100% accuracy? What do you need to improve?
– GGA
Jan 27, 2017 at 7:48
• You could limit the coefficients of the linear model to a base of 0.1 and a maximum of 0.9, effectively saying that you trust each classifier a little (min 0.1) and no classifier completely (max 0.9). Jan 27, 2017 at 8:47
• Use the validated statistics, training statistics are useless. Apr 4, 2017 at 0:22