What does it mean to "fit a regression function" and then use it to update other functions? Referring to the algorithm on page 11 in this paper on boosting algorithms, I really don't understand step 2, (ii) and (iii).
What does this mean: 

(ii) Fit the regression function $g_j^h (x)$ by weighted least-squares of
  the working response $z_{ij}$ to $x_i$ with weights $w_{ij}$ on the training
  data.
(iii) Set $g_j(x) \leftarrow g_j(x) + g_j^h (x)$

For convenience, I'm pasting the algorithm here:

I do understand the theory and general concept of least squares regression. I understand we have data whose graph doesn't have a clear line or curve formula, and so we try to come up with a function that'll help us predict a response given an input. Using "least-squares" means coming up with a formula whose parameters are such that the square of the errors of the prediction is minimized. 
But once this regression function is "fitted" to data, how is it "added" to other functions like itself?
I'm having a hard time understanding the application of the above quoted statement. Not just in this algorithm, but in others, wherever they mention "fitting" a regression function and then using that function with others. At best I understand using regression functions to actually predict values as output. But how do we use them to predict/generate more functions?
Could someone please explain this simply, maybe using arrays of data or something?
 A: You have to notice that you not use in this algorithm the "least-squares", but "weighted least squares". See Wikipedia page dedicated to this matter. 
The main distinction is that instead of the usual:
$$LSE(y, \hat{y}) = E[(y - \hat{y})^2] = E[(y-\hat{f}(x, \beta))^2]$$
you have an additional factor
$$WLSE(y, \hat{y}) = E[(y - w\hat{y})^2] = E[(y-w \hat{f}(x,\beta))^2]$$
where $w$ comes from weights. In the standard weighted least squares, weights have some properties, to be able to get a BLUE estimator. In your case the weights are the current weights computed at step $(i)$.
Now you have also to note that at step $(b)$ the predictor for the current step $g_j(x)$ is altered by adding previously fitted predictors. Note that the formula from the $(b)$ step can be decomposed as a linear combination of other functions. Which means it has the form:
$$g(x) = \beta_0 + \beta_1f_1(x) + \beta_2f_2(x) .. + \beta_pf_p(x)$$
where $\beta_i$ are constant factors, and $f_i$are functions of input variables.
Thus, in the end, the final computed $g_j(x)$ will be a linear combination of the fitted function at step $j$ and all other previously fitted predictors.  
This qualifies the final $g_j(x)$ to be an additive model since an generalized additive model is a linear combination of other predictors. 
