Given a RS $X_{1},X_{2},\ldots,X_{n}$ whose distribution is well known (unless its parameters), how do we prove the following Fischer Information relationship \begin{align*} I_{F}(\theta) =\textbf{E}\left[\left(\frac{\partial\ln f(\textbf{X}|\theta)}{\partial\theta}\right)^{2}\right] = -\textbf{E}\left[\frac{\partial^{2}\ln f(\textbf{X}|\theta)}{\partial\theta^{2}}\right] \end{align*}


To begin with, I started with the observation that \begin{align*} &\frac{\partial^{2}\ln f(\textbf{X}|\theta)}{\partial\theta^{2}} = \frac{\partial}{\partial\theta}\left[\frac{\partial\ln f(\textbf{X}|\theta)}{\partial\theta}\right] = \frac{\partial}{\partial\theta}\left[\frac{1}{f(\textbf{X}|\theta)}\frac{\displaystyle\partial f(\textbf{X}|\theta)}{\displaystyle\partial \theta}\right] = \ldots \end{align*}

But I do not know how to proceed from here. Can someone help me out? Thanks in advance!


Following your way, calling $f(\textbf{X}|\theta)=f$, for notational simplicity: $$\begin{align}\frac{\partial^2 \ln f}{\partial\theta^2}&=\frac{\partial}{\partial\theta}\left(\frac{\partial\ln f}{\partial\theta}\right)=\frac{\partial}{\partial\theta}\left(\frac{1}{f}\frac{\partial f}{\partial\theta}\right)\\&=-\frac{1}{f^2}\left(\frac{\partial f}{\partial\theta}\right)^2+\frac{1}{f}\frac{\partial^2 f}{\partial\theta^2}\end{align}$$

Note that the first summand, call it $\mathcal{F}$, can be manipulated as: $$\mathcal{F}=-\left(\frac{1}{f}\frac{\partial f}{\partial\theta}\right)^2=-\left(\frac{\partial\ln f}{\partial\theta}\right)^2$$ because inner term is the same with the rightmost term we found in the first line.

For the second summand, we have: $$E\left[\frac{1}{f}\frac{\partial^2 f}{\partial\theta^2}\right]=\int \frac{\partial^2 f}{\partial\theta^2}dX=\frac{\partial^2}{\partial\theta^2}\left(\int fdX\right)=\frac{\partial^2 f}{\partial\theta^2}(1)=0$$ So, only the first summand remains non-zero, which is what we need to find. Note that. in order for this equation to be true, the differentiation by $\theta$ needs to be drawn out of the integral via assumed regularity conditions.

| cite | improve this answer | |
  • $\begingroup$ The notation is brief to the point of being (potentially) confusing. Does "$dX$" mean "$f(x)dx$"? How do you obtain the last two inequalities (neither of which seem correct)? $\endgroup$ – whuber Apr 23 '19 at 19:45
  • $\begingroup$ $dX$ is as is because $$E\left[\frac{1}{f}\frac{\partial^2 f}{\partial\theta^2}\right]=\int \frac{1}{f(X|\theta)}\frac{\partial^2 f(X|\theta)}{\partial\theta^2}f(X|\theta)dX=\int \frac{\partial^2 f(X|\theta)}{\partial\theta^2}dX$$. Which one is the other confusing one? The equality for $\mathcal{F}$? $\endgroup$ – gunes Apr 23 '19 at 20:01
  • $\begingroup$ It appears you have evaluated "$\int f dX$" as "$f(1)$" rather than as $1.$ The use of a capital $X$ to denote both a random variable and variable of integration technically works but is potentially confusing. $\endgroup$ – whuber Apr 23 '19 at 21:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.