enter image description here

Hi, I am reading something about Boosting, and I had hard time understanding one of the steps in boosting - assign greater weights to those instances.

What does the sentence - assign greater weights to those instances mean ? My understanding is.. for example

Initially, we have training data ($x_1$,$y_1$) ($x_2$,$y_2$) ($x3$,$y_3$) ($x_4$,$y_4$) ($x_5$,$y_5$) after we first apply the weak learner, we find that (x2,y2) (x3,y3) are misclassified, and we try to adjust the training data, "assigning the weights" so that new training data become... like ($x_1$,$y_1$) ($x_2$,$y_2$),($x_2$,$y_2$) ($x_3$,$y_3$) ($x_3$,$y_3$) ($x_4$,$y_4$) ($x_5$,$y_5$), where we have more misclassified instances ? thus, next learner have more chances to learn the misclassified ones ?


"Instance" is just a somewhat confusing way of saying "case" or "person" or "observation," etc.

Imagine we have N data points we are trying to predict; each of those data points would be an "instance." If our data look like:

  y x
1 1 4
2 0 2
3 0 3
4 1 3
5 1 3

Then we have 5 "instances" and each row (observation, case, etc.) represents an instance. Imagine we predict y from x using a weak learner. We find that instance #3 (y = 0, x = 3) is classified incorrectly. In the next iteration, we would weight that instance higher than the others.

I wouldn't necessarily say that the learner "has more chances to learn the misclassified ones," as every instance/case/row/observation is included in each iteration. It is just that subsequent learners focus more on misclassified instances.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.