# Model stacking, what is the input of meta classifier?

I know that by stacking different models among which there has a low correlation can boost the performance of on single model. And I found a picture

In step 7, the $h_j(x_i)$ in new data $x_i^{'}=\{h_1(x_i), h_2(x_i), ..., h_T(x_i)\}$ is the output class label or the probability of model $j$?

## 1 Answer

It can be either (or both), but in practice, the columns of the level-one matrix are the the predicted values (not the class labels) for each base learner.

Side note: Keep in mind that unless the model is calibrated using something like Platt scaling, that the output of a classification model won't be a "probability", it's just some numeric predicted value. It's often assumed to be, or referred to, as a probability because the value is between 0 and 1.

• In regression, it's simply the predicted value.
• In binary classification, it's the predicted value for the positive class. In theory, for each base learner, you could add both columns (predicted value for negative class & predicted value for positive class) to the level-one matrix, but most stacking implementations (e.g. H2O Stacked Ensemble, SuperLearner) use just use the predicted value for the positive class.
• In multiclass classification, it's typically the set of predicted values for all the classes. In other words, if you had 5 base learners and 3 classes, the number of columns in the level-one matrix will be 5*3=15, plus the response column.