Boosting for regression systems

Question

I am quite a newbie in this area:

• What are the boosting methods for regression systems? I know about Gradient boosting; are there any other approaches?
• Are there textbooks or tutorials devoted to this area?

Answer 1

While I haven't seen anything specifically, I doubt that it would achieve much, at least for linear regression. Each regression equation is just a linear combination of the predictors: $$\hat{y}^{(j)} = \sum_i \hat{\beta}_i^{(j)} x_i$$ Most boosting algorithms in turn combine multiple predictors by taking a weighted average, which is another linear combination: $$\hat{y} = \sum_j \hat{w}^{(j)} \hat{y}^{(j)}$$ So the final output is still a linear combination of the predictors, only with the coefficients obtained by a complicated process.