# Show that solving this equation is the same as m'th Fisher Scoring step

In my exercise we assume that $$Y_i|X_i$$ has distribution with density $$f_i(y_i,\eta_i)$$ for $$i=1,...,n$$ where $$\eta_i=X_i^T$$ is the linear predictor. The generalized linear model with an exponential dispersion distribution is a case with: $$f_i(y_i,\eta_i)=e^{\frac{\theta(\eta_i)y_i-c(\eta_i)}{\Psi}}$$ It is assumed that assumed that $$f_i(y_i, \eta_i)>0$$ and we define that: $$U_i(\eta_i)=\frac{\partial}{\partial \eta_i}log f_i(Y_i, \eta_i)$$ and $$w_i=-E(\frac{\partial^2}{\partial^2 \eta_i}log f_i(Y_i, \eta_i))$$

We introduce now the score vector $$U(\eta)=(U_1(\eta_1),..., U_n(\eta_n))^T$$ and the diagonal matrix W with $$W_{ii}=w_i$$

I have to prove that for the m'th Fisher Scoring step for solving the score equation amounts to solving the linear equation: $$XU(\mu_m)-(X^TW_mX)(\beta-\beta_m)=0$$ Where here $$W_m$$ is defined in terms of $$\eta=X\beta_m$$.

I think that there by Fisher Scoring step means in the Fisher Scoring Algoritm? And I have found in my book that for the Fisher Scoring Algoritm: $$\beta_{m+1}=\beta_m+(X^TW_mX)^{-1}X^TU(\eta_m)$$ But how can I show it's same solving the two equations?

See this previous question

Score statistic and Fisher information

• can you Taylor expand? Nov 2, 2021 at 19:52
• Yes have learned about Taylor expand but on simple examples. Can I use Taylor Expand to show it? Nov 2, 2021 at 19:54
• Can you help me? Nov 2, 2021 at 20:48

I actually do not know how to do this exactly, but you can write $$U(\beta)\approx U(\beta_0)+(\beta-\beta_0)U'(\beta_0)$$ (Taylor expansion). Note the similarity to your second to last display. Now take this and solve for $$(\beta-\beta_0)$$ and pull $$\beta_0$$ onto the other side, and note the similarity to your last display. You must show that the solution to those two displays is the same, and this seems to be a way to connect them.
• Now it makes a bit sense with the Taylor Expand. But I'm not totally sure. You use that $\eta_m=X\beta_m$ and Taylor expaned and get $𝑈(\beta)≈𝑈(\beta_0)+(\beta−\beta_0)𝑈′(\beta_0)$, it makes sense, and when isolating $\beta$ I get that $\beta=\beta_0+\frac{U(\beta)-U(\beta_0)}{U'(\beta_0)}$. It seems a bit similar to my Fisher Scoring Algoritm equation, but it's not totally clear for my. Can you help showing that is the same? Nov 3, 2021 at 10:11
• One thing is that if $\beta_n$ minimizes $U,$ $U(\beta_n)=0.$ Hence replace $\beta$ with $\beta_n$ in the Taylor expansion, and you will get rid of the term $U(\beta_n).$ Nov 3, 2021 at 13:35
• Note also that something like $X^TX/n$ goes in probability usually to the expectation of the second derivative of $U$ Nov 3, 2021 at 13:36
• I think you are getting closer. My strategy would be to do this expansion, etc for a simpler function like a univariate $f(x)=x^2$ and then see how maybe the Taylor expansion works with Fisher scoring. Then add randomness to $f(x).$ Nov 3, 2021 at 13:41