Show condition so $\beta_{m+1}$ is the solution of weighted least squares problem

Question

In my exercise we assume that $Y_i|X_i$ has distribution with density $f_i(y_i,\eta_i) $ for $i=1,...,n$ where $\eta_i=X_i^T$ is the linear predictor. The generalized linear model with an exponential dispersion distribution is a case with: $$f_i(y_i,\eta_i)=e^{\frac{\theta(\eta_i)y_i-c(\eta_i)}{\Psi}}$$ It is assumed that assumed that $f_i(y_i, \eta_i)>0$ and we define that: $$U_i(\eta_i)=\frac{\partial}{\partial \eta_i}log f_i(Y_i, \eta_i)$$ and $$w_i=-E(\frac{\partial^2}{\partial^2 \eta_i}log f_i(Y_i, \eta_i))$$

We introduce now the score vector $U(\eta)=(U_1(\eta_1),..., U_n(\eta_n))^T$ and the diagonal matrix W with $W_{ii}=w_i$

I have that for the m'th Fisher Scoring step(fisher scoring algoitm) for solving the score equation amounts to solving the linear equation: $$XU(\mu_m)-(X^TW_mX)(\beta-\beta_m)=0$$ Where here $W_m$ is defined in terms of $\eta=X\beta_m$. in my book that for the Fisher Scoring Algoritm equation: $$\beta_{m+1}=\beta_m+(X^TW_mX)^{-1}X^TU(\eta_m)$$

Now I have to show that if $w_{m,i}\geq 0$ then $\beta_{m+1}$ is the solution of weighted least squares problem with diagonal weight matrix $W_m$ and working responses $X_i^T\beta_m+U_i(\eta_m)/w_{m,j}$.

I'm not totally into the Fisher Scoring algoritm. Can anyone help me with this problem and how to set it up? Working responses what does that mean?

Answer 1

the solution of weighted least squares problem with diagonal weight matrix $W_m$ and working responses $X_i^T\beta_m+U_i(\eta_m)/w_{m,j}$.

...

Working responses what does that mean?

The working response is the response $Y_{m,i}$ in the weighted least squares problem during the current step

$$\min_{\beta_{m+1}} W_{m,i} (Y_{m,i} - X_i^T\beta_{m+1})^2 $$

This response $Y_{m,i}$ is a transformed version of the current estimate.

$$Y_{m,i} = X_i^T\beta_m+U_i(\eta_m)/w_{m,j}$$

What you really want to solve is the least-squares problem with the transformed variable for the actual solution

$$Y_{\infty,i} = X_i^T\beta_\infty+U_i(\eta_\infty)/w_{\infty,j}$$

but since you do not know the solution $\beta_\infty$ you plugin the current solution $\beta_m$ and that is what you are 'working' with.