# Boosting reduces bias when compared to what algorithm?

I am reading on bagging and boosting, and I understand how they both work (at least I think I do). I would like to talk in the context of decision tree ensembles as I think (not sure if correct) that these are the most used when it comes to bagging and boosting.

It is said that bagging reduces variance and boosting reduces bias.

Now, I understand why bagging would reduce variance of a decision tree algorithm, since on their own, decision trees are low bias high variance, and when we make an ensemble of them with bagging, we reduce the variance as we now spread the vote (classification) or average over (regression) all of them. (Could somebody please point out if this is incorrect).

But I don't understand why it is said that boosting reduces bias. Exactly of what algorithm it reduces the bias of? Because if we compare it to a decision tree, then surely the decision tree can have 0 bias when we classify each point and have 0 error, so how does it reduce the bias? If we compare it to a bagging algorithm, then it does actually have less bias than the bagging algorithm since now we're incorporating the whole dataset and we're also focusing on all data points incorrectly classified.

So when we say that boosting reduces bias, do we say this when we compare it to the bagging algorithm or to something else?

• The base algorithm, the one being boosted that is. Commented Nov 15, 2021 at 22:11
• @jbowman but how can a single decision tree be more biased than boosting if it has $0$ training error? Commented Nov 15, 2021 at 22:28