1
$\begingroup$

I know that by stacking different models among which there has a low correlation can boost the performance of on single model. And I found a picture enter image description here

In step 7, the $h_j(x_i)$ in new data $x_i^{'}=\{h_1(x_i), h_2(x_i), ..., h_T(x_i)\}$ is the output class label or the probability of model $j$?

$\endgroup$
1
$\begingroup$

It can be either (or both), but in practice, the columns of the level-one matrix are the the predicted values (not the class labels) for each base learner.

Side note: Keep in mind that unless the model is calibrated using something like Platt scaling, that the output of a classification model won't be a "probability", it's just some numeric predicted value. It's often assumed to be, or referred to, as a probability because the value is between 0 and 1.

  • In regression, it's simply the predicted value.
  • In binary classification, it's the predicted value for the positive class. In theory, for each base learner, you could add both columns (predicted value for negative class & predicted value for positive class) to the level-one matrix, but most stacking implementations (e.g. H2O Stacked Ensemble, SuperLearner) use just use the predicted value for the positive class.
  • In multiclass classification, it's typically the set of predicted values for all the classes. In other words, if you had 5 base learners and 3 classes, the number of columns in the level-one matrix will be 5*3=15, plus the response column.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.