I'm learning about boosting. I think I understand how adaptive boosting works for classification. I'm trying to get some intuition for regression boosting.
At each iteration, adaptive boosting forces the next weak learner to focus more on incorrectly classified points. Intuitively, I can see why that should lead to a good classifier. I could be wrong, but L2 boosting doesn't seem to do anything like that. In L2 boosting, at each iteration, you're fitting a weak learner to the previous iteration's residuals. In a regression tree, when you're splitting a node, aren't you "fitting" the residuals from that node and that node's parents? In both cases, you're "fitting" unweighted residuals, so I don't understand why they're different.
Maybe the the main advantage of L2 boosting over a single tree is that the former has many more regularization/bootstrap-ish options (e.g., randomly choosing subsets of features, the learning rate, the number of trees, individual tree depth, etc.)?