# Boosted Trees: Objective Function clarification

Reading through this overview of boosted trees, I'm having trouble understanding how the second line was derived.

$$Obj(t)=\sum_1^n{loss(y_{i} - \hat{y}_i^{(t)})} + \sum_1^t{\Omega(f_i)} \\ = \sum_1^n{loss(y_{i} - (\hat{y}_i^{(t-1)} + f_t(x_i)))} + \Omega(f_t) + Constant$$

Why do we ignore the regularization of the other functions? And is the "Constant" term the irreducible error?

Also, in the equation following the one above, we use MSE as the loss function which results in:

$$Obj(t)=\sum_1^n{(y_{i} - (\hat{y}_i^{(t-1)} + f_t(x_i))^2} + \sum_1^t{\Omega(f_i)} \\ = \sum_1^n{[2(\hat{y}_i^{(t-1)} - y_i)f_t(x_i) + f_t(x_i)^2]} + \Omega(f_t) + Constant$$

Why does the $( \hat{y}_i^{(t-1)} - y_i )^2$ term disappear in the expansion?

$$\hat y_i^{(t)} = y_{i}^{(t-1)} + f_t(x_i)$$
Observe that, at this point in the discussion, we are attempting to fit the weak learner $f_t$, and all the previous weak learners $f_{t-1}, f_{t-2}, \ldots$ have already been fit, and are hence fixed. This means that any function of them, in particular $\Omega(f_{i-1}), \Omega(f_{i-2}), \ldots$ are also fixed. So breaking up the sum
$$\sum_1^t \Omega(f_i) = \underbrace{\Omega(f_t)}_{\text{Currently up for optimization}} + \underbrace{\sum_1^{t-1} \Omega(f_t)}_{\text{Already fixed}}$$
• I'll add it to my answer in detail, but quickly, that term is absorbed into "constant" for the same reason, as $\hat y^{(t-1)}_i$ is fixed when you are fitting the $t$'th weak learner. – Matthew Drury Jan 22 '16 at 16:22