1
$\begingroup$

In my exercise we assume that $Y_i|X_i$ has distribution with density $f_i(y_i,\eta_i) $ for $i=1,...,n$ where $\eta_i=X_i^T$ is the linear predictor. The generalized linear model with an exponential dispersion distribution is a case with: $$f_i(y_i,\eta_i)=e^{\frac{\theta(\eta_i)y_i-c(\eta_i)}{\Psi}}$$ It is assumed that assumed that $f_i(y_i, \eta_i)>0$ and we define that: $$U_i(\eta_i)=\frac{\partial}{\partial \eta_i}log f_i(Y_i, \eta_i)$$ and $$w_i=-E(\frac{\partial^2}{\partial^2 \eta_i}log f_i(Y_i, \eta_i))$$

We introduce now the score vector $U(\eta)=(U_1(\eta_1),..., U_n(\eta_n))^T$ and the diagonal matrix W with $W_{ii}=w_i$

I have to prove that for the m'th Fisher Scoring step for solving the score equation amounts to solving the linear equation: $$XU(\mu_m)-(X^TW_mX)(\beta-\beta_m)=0$$ Where here $W_m$ is defined in terms of $\eta=X\beta_m$.

I think that there by Fisher Scoring step means in the Fisher Scoring Algoritm? And I have found in my book that for the Fisher Scoring Algoritm: $$\beta_{m+1}=\beta_m+(X^TW_mX)^{-1}X^TU(\eta_m)$$ But how can I show it's same solving the two equations?

See this previous question

Score statistic and Fisher information

Improve this question
$\endgroup$
3
  • $\begingroup$ can you Taylor expand? $\endgroup$
    – Sam Weisenthal
    Nov 2, 2021 at 19:52
  • $\begingroup$ Yes have learned about Taylor expand but on simple examples. Can I use Taylor Expand to show it? $\endgroup$
    – Lifeni
    Nov 2, 2021 at 19:54
  • $\begingroup$ Can you help me? $\endgroup$
    – Lifeni
    Nov 2, 2021 at 20:48

1 Answer 1

Reset to default
0
$\begingroup$

I actually do not know how to do this exactly, but you can write $U(\beta)\approx U(\beta_0)+(\beta-\beta_0)U'(\beta_0)$ (Taylor expansion). Note the similarity to your second to last display. Now take this and solve for $(\beta-\beta_0)$ and pull $\beta_0$ onto the other side, and note the similarity to your last display. You must show that the solution to those two displays is the same, and this seems to be a way to connect them.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Now it makes a bit sense with the Taylor Expand. But I'm not totally sure. You use that $\eta_m=X\beta_m$ and Taylor expaned and get $𝑈(\beta)≈𝑈(\beta_0)+(\beta−\beta_0)𝑈′(\beta_0)$, it makes sense, and when isolating $\beta$ I get that $\beta=\beta_0+\frac{U(\beta)-U(\beta_0)}{U'(\beta_0)}$. It seems a bit similar to my Fisher Scoring Algoritm equation, but it's not totally clear for my. Can you help showing that is the same? $\endgroup$
    – Lifeni
    Nov 3, 2021 at 10:11
  • $\begingroup$ One thing is that if $\beta_n$ minimizes $U,$ $U(\beta_n)=0.$ Hence replace $\beta$ with $\beta_n$ in the Taylor expansion, and you will get rid of the term $U(\beta_n).$ $\endgroup$
    – Sam Weisenthal
    Nov 3, 2021 at 13:35
  • $\begingroup$ Note also that something like $X^TX/n$ goes in probability usually to the expectation of the second derivative of $U$ $\endgroup$
    – Sam Weisenthal
    Nov 3, 2021 at 13:36
  • $\begingroup$ I think you are getting closer. My strategy would be to do this expansion, etc for a simpler function like a univariate $f(x)=x^2$ and then see how maybe the Taylor expansion works with Fisher scoring. Then add randomness to $f(x).$ $\endgroup$
    – Sam Weisenthal
    Nov 3, 2021 at 13:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.