5
$\begingroup$

Given a RS $X_{1},X_{2},\ldots,X_{n}$ whose distribution is well known (unless its parameters), how do we prove the following Fischer Information relationship \begin{align*} I_{F}(\theta) =\textbf{E}\left[\left(\frac{\partial\ln f(\textbf{X}|\theta)}{\partial\theta}\right)^{2}\right] = -\textbf{E}\left[\frac{\partial^{2}\ln f(\textbf{X}|\theta)}{\partial\theta^{2}}\right] \end{align*}

MY APPROACH

To begin with, I started with the observation that \begin{align*} &\frac{\partial^{2}\ln f(\textbf{X}|\theta)}{\partial\theta^{2}} = \frac{\partial}{\partial\theta}\left[\frac{\partial\ln f(\textbf{X}|\theta)}{\partial\theta}\right] = \frac{\partial}{\partial\theta}\left[\frac{1}{f(\textbf{X}|\theta)}\frac{\displaystyle\partial f(\textbf{X}|\theta)}{\displaystyle\partial \theta}\right] = \ldots \end{align*}

But I do not know how to proceed from here. Can someone help me out? Thanks in advance!

$\endgroup$

1 Answer 1

4
$\begingroup$

Following your way, calling $f(\textbf{X}|\theta)=f$, for notational simplicity: $$\begin{align}\frac{\partial^2 \ln f}{\partial\theta^2}&=\frac{\partial}{\partial\theta}\left(\frac{\partial\ln f}{\partial\theta}\right)=\frac{\partial}{\partial\theta}\left(\frac{1}{f}\frac{\partial f}{\partial\theta}\right)\\&=-\frac{1}{f^2}\left(\frac{\partial f}{\partial\theta}\right)^2+\frac{1}{f}\frac{\partial^2 f}{\partial\theta^2}\end{align}$$

Note that the first summand, call it $\mathcal{F}$, can be manipulated as: $$\mathcal{F}=-\left(\frac{1}{f}\frac{\partial f}{\partial\theta}\right)^2=-\left(\frac{\partial\ln f}{\partial\theta}\right)^2$$ because inner term is the same with the rightmost term we found in the first line.

For the second summand, we have: $$E\left[\frac{1}{f}\frac{\partial^2 f}{\partial\theta^2}\right]=\int_X \frac{\partial^2 f}{\partial\theta^2}dx=\frac{\partial^2}{\partial\theta^2}\left(\int_X fdx\right)=\frac{\partial^2}{\partial\theta^2}(1)=0$$ So, only the first summand remains non-zero, which is what we need to find. Note that. in order for this equation to be true, the differentiation by $\theta$ needs to be drawn out of the integral via assumed regularity conditions.

$\endgroup$
7
  • $\begingroup$ The notation is brief to the point of being (potentially) confusing. Does "$dX$" mean "$f(x)dx$"? How do you obtain the last two inequalities (neither of which seem correct)? $\endgroup$
    – whuber
    Apr 23, 2019 at 19:45
  • $\begingroup$ $dX$ is as is because $$E\left[\frac{1}{f}\frac{\partial^2 f}{\partial\theta^2}\right]=\int \frac{1}{f(X|\theta)}\frac{\partial^2 f(X|\theta)}{\partial\theta^2}f(X|\theta)dX=\int \frac{\partial^2 f(X|\theta)}{\partial\theta^2}dX$$. Which one is the other confusing one? The equality for $\mathcal{F}$? $\endgroup$
    – gunes
    Apr 23, 2019 at 20:01
  • $\begingroup$ It appears you have evaluated "$\int f dX$" as "$f(1)$" rather than as $1.$ The use of a capital $X$ to denote both a random variable and variable of integration technically works but is potentially confusing. $\endgroup$
    – whuber
    Apr 23, 2019 at 21:16
  • $\begingroup$ Hi@gunes, How can I link this:"first summand non-zero" and "second zero" to fisher information? Thanks $\endgroup$
    – Yiffany
    Aug 14, 2023 at 12:19
  • 1
    $\begingroup$ Hi @Yiffany, when the second summand is zero, the first summand in the expectation formula becomes what the OP was asking, i.e. the first equation in the OP, LHS. $\endgroup$
    – gunes
    Aug 14, 2023 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.