4
$\begingroup$

I am trying to apply the following definition of Fisher Information: \begin{align*} \mathcal{I}_{1}(\theta) = \mathbb{E}_{\theta}\left[\left(\frac{\partial}{\partial\theta}\ln f(x_{1}|\theta)\right)^{2}\right] \end{align*}

But I have not succeeded so far. This is my attempt so far: \begin{align*} \mathcal{I}_{1}(\eta) & = \mathbb{E}_{\eta}\left[\left(\frac{\partial}{\partial\eta}\ln f(x_{1}|\eta)\right)^{2}\right]\\\\ & = \mathbb{E}_{\eta}\left[\left(\frac{\partial\theta}{\partial\eta}\frac{\partial}{\partial\theta}\ln f(x_{1}|\eta)\right)^{2}\right] \end{align*}

Can someone help me to finish the demonstration?

$\endgroup$

2 Answers 2

4
$\begingroup$

It's just a consequence of the chain rule. The density of $X$ is $f(x; \theta) = f(x; h(\eta))$. Therefore, viewing it as a function of $\eta$, we have \begin{align} \frac{d\ln f(x; h(\eta))}{d\eta} = \frac{1}{f(x; h(\eta))}\frac{df(x; \theta)}{d\theta}\frac{d\theta}{d\eta} = \frac{1}{f(x; \theta)}\frac{df(x; \theta)}{d\theta}h'(\eta) = \frac{d\ln f(x; \theta)}{d\theta}h'(\eta). \end{align}

It then follows that \begin{align} I(\eta) &= \int_{\mathbb{R}}\left[\frac{d\ln f(x; h(\eta))}{d\eta}\right]^2f(x;h(\eta))dx \\ &= \int_{\mathbb{R}}\left[\frac{d\ln f(x; \theta)}{d\theta}h'(\eta)\right]^2f(x; \theta)dx \\ &= [h'(\eta)]^2\int_{\mathbb{R}}\left[\frac{d\ln f(x; \theta)}{d\theta}\right]^2f(x; \theta)dx \\ &= [h'(\eta)]^2I(\theta). \end{align}

$\endgroup$
5
$\begingroup$

Start out from likelihood function $L(\theta;Y_1)$ (for a single observation) and make the reparametrization $\theta = h(\eta)$ to get the reparametrized log-likelihood $L(\eta;Y_1) = L(h(\eta);Y_1)$. Then the log-likelihood function of $\eta$ is $\ell(\eta;Y_1) = \ell(h(\eta);Y_1)$ so the Fisher information for a single observation is

\begin{align*} I_1(\eta) & = E_\theta\left[\left(\frac{d \ell(\eta;Y_1)}{d\eta}\right)^2\right]\\ & = E_\theta\left[\left(\frac{d \ell(h(\eta;Y_1))}{d h(\eta)}\frac{dh(\eta)}{d\eta}\right)^2\right]\\ & = E_\theta\left[\left(\frac{d \ell(\theta;Y_1)}{d \theta}\right)^2\right]\left(\frac{dh(\eta)}{d\eta}\right)^2\\ & = I_1(\theta)\left(\frac{dh(\eta)}{d\eta}\right)^2. \end{align*}

$\endgroup$
1
  • 1
    $\begingroup$ That is neat and precise. +1. $\endgroup$ Nov 18, 2022 at 23:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.