# How do we deduce this fisher information relation?

Given a RS $$X_{1},X_{2},\ldots,X_{n}$$ whose distribution is well known (unless its parameters), how do we prove the following Fischer Information relationship \begin{align*} I_{F}(\theta) =\textbf{E}\left[\left(\frac{\partial\ln f(\textbf{X}|\theta)}{\partial\theta}\right)^{2}\right] = -\textbf{E}\left[\frac{\partial^{2}\ln f(\textbf{X}|\theta)}{\partial\theta^{2}}\right] \end{align*}

MY APPROACH

To begin with, I started with the observation that \begin{align*} &\frac{\partial^{2}\ln f(\textbf{X}|\theta)}{\partial\theta^{2}} = \frac{\partial}{\partial\theta}\left[\frac{\partial\ln f(\textbf{X}|\theta)}{\partial\theta}\right] = \frac{\partial}{\partial\theta}\left[\frac{1}{f(\textbf{X}|\theta)}\frac{\displaystyle\partial f(\textbf{X}|\theta)}{\displaystyle\partial \theta}\right] = \ldots \end{align*}

But I do not know how to proceed from here. Can someone help me out? Thanks in advance!

Following your way, calling $$f(\textbf{X}|\theta)=f$$, for notational simplicity: \begin{align}\frac{\partial^2 \ln f}{\partial\theta^2}&=\frac{\partial}{\partial\theta}\left(\frac{\partial\ln f}{\partial\theta}\right)=\frac{\partial}{\partial\theta}\left(\frac{1}{f}\frac{\partial f}{\partial\theta}\right)\\&=-\frac{1}{f^2}\left(\frac{\partial f}{\partial\theta}\right)^2+\frac{1}{f}\frac{\partial^2 f}{\partial\theta^2}\end{align}

Note that the first summand, call it $$\mathcal{F}$$, can be manipulated as: $$\mathcal{F}=-\left(\frac{1}{f}\frac{\partial f}{\partial\theta}\right)^2=-\left(\frac{\partial\ln f}{\partial\theta}\right)^2$$ because inner term is the same with the rightmost term we found in the first line.

For the second summand, we have: $$E\left[\frac{1}{f}\frac{\partial^2 f}{\partial\theta^2}\right]=\int_X \frac{\partial^2 f}{\partial\theta^2}dx=\frac{\partial^2}{\partial\theta^2}\left(\int_X fdx\right)=\frac{\partial^2}{\partial\theta^2}(1)=0$$ So, only the first summand remains non-zero, which is what we need to find. Note that. in order for this equation to be true, the differentiation by $$\theta$$ needs to be drawn out of the integral via assumed regularity conditions.

• The notation is brief to the point of being (potentially) confusing. Does "$dX$" mean "$f(x)dx$"? How do you obtain the last two inequalities (neither of which seem correct)?
– whuber
Apr 23, 2019 at 19:45
• $dX$ is as is because $$E\left[\frac{1}{f}\frac{\partial^2 f}{\partial\theta^2}\right]=\int \frac{1}{f(X|\theta)}\frac{\partial^2 f(X|\theta)}{\partial\theta^2}f(X|\theta)dX=\int \frac{\partial^2 f(X|\theta)}{\partial\theta^2}dX$$. Which one is the other confusing one? The equality for $\mathcal{F}$? Apr 23, 2019 at 20:01
• It appears you have evaluated "$\int f dX$" as "$f(1)$" rather than as $1.$ The use of a capital $X$ to denote both a random variable and variable of integration technically works but is potentially confusing.
– whuber
Apr 23, 2019 at 21:16
• Hi@gunes, How can I link this:"first summand non-zero" and "second zero" to fisher information? Thanks Aug 14, 2023 at 12:19
• Hi @Yiffany, when the second summand is zero, the first summand in the expectation formula becomes what the OP was asking, i.e. the first equation in the OP, LHS. Aug 14, 2023 at 19:10