2
$\begingroup$

In my exercise we assume that $Y_i|X_i$ has distribution with density $f_i(y_i,\eta_i) $ for $i=1,...,n$ where $\eta_i=X_i^T$ is the linear predictor. The generalized linear model with an exponential dispersion distribution is a case with: $$f_i(y_i,\eta_i)=e^{\frac{\theta(\eta_i)y_i-c(\eta_i)}{\Psi}}$$ It is assumed that assumed that $f_i(y_i, \eta_i)>0$ and we define that: $$U_i(\eta_i)=\frac{\partial}{\partial \eta_i}log f_i(Y_i, \eta_i)$$ and $$w_i=-E(\frac{\partial^2}{\partial^2 \eta_i}log f_i(Y_i, \eta_i))$$

We introduce now the score vector $U(\eta)=(U_1(\eta_1),..., U_n(\eta_n))^T$ and the diagonal matrix W with $W_{ii}=w_i$

I have that for the m'th Fisher Scoring step(fisher scoring algoitm) for solving the score equation amounts to solving the linear equation: $$XU(\mu_m)-(X^TW_mX)(\beta-\beta_m)=0$$ Where here $W_m$ is defined in terms of $\eta=X\beta_m$. in my book that for the Fisher Scoring Algoritm equation: $$\beta_{m+1}=\beta_m+(X^TW_mX)^{-1}X^TU(\eta_m)$$

Now I have to show that if $w_{m,i}\geq 0$ then $\beta_{m+1}$ is the solution of weighted least squares problem with diagonal weight matrix $W_m$ and working responses $X_i^T\beta_m+U_i(\eta_m)/w_{m,j}$.

I'm not totally into the Fisher Scoring algoritm. Can anyone help me with this problem and how to set it up? Working responses what does that mean?

$\endgroup$

1 Answer 1

1
$\begingroup$

the solution of weighted least squares problem with diagonal weight matrix $W_m$ and working responses $X_i^T\beta_m+U_i(\eta_m)/w_{m,j}$.

...

Working responses what does that mean?

The working response is the response $Y_{m,i}$ in the weighted least squares problem during the current step

$$\min_{\beta_{m+1}} W_{m,i} (Y_{m,i} - X_i^T\beta_{m+1})^2 $$

This response $Y_{m,i}$ is a transformed version of the current estimate.

$$Y_{m,i} = X_i^T\beta_m+U_i(\eta_m)/w_{m,j}$$

What you really want to solve is the least-squares problem with the transformed variable for the actual solution

$$Y_{\infty,i} = X_i^T\beta_\infty+U_i(\eta_\infty)/w_{\infty,j}$$

but since you do not know the solution $\beta_\infty$ you plugin the current solution $\beta_m$ and that is what you are 'working' with.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.