Suppose $X$ is a one dimensional Normal variable.

Find the $2 \times 2$ fisher information matrix $I(\theta)$ for $\theta = (E(X), E(X^2))$.

Basically, what I did was to write down the pdf in terms of $E(X)$ and $E(X^2)$...and so I got

Let $d = E(X^2)$ (so $\sigma^2 = d - \mu^2$)

$(\frac{1}{2(d-\mu^2)})^{1/2} e^{[-(X-\mu)^2]/[2(d-\mu^2)]}$.

And then I tried to compute the matrix from that in the usual way....but it is literally taking me forever to compute this matrix (especially when it comes to computing the first and second derivative of $\log f(x,\mu, d)$, with respect to $\mu$). So I was just wondering if there was an easier approach for this problem.

I do know that the fisher matrix for the normal distribution with $\theta = (\mu ,\sigma)$ is $$\begin{bmatrix} 1/\sigma^2 & 0 \\ 0 & 2/\sigma^2 \end{bmatrix}$$

So I was wondering if there was any way to manipulate this to get the fisher matrix when the parameter is $\theta = (E(x), E(x^2))$.

Thanks in advance


Since what you search involves components of the variance and not of the standard deviation, start with the column vector $ \theta = (\mu'_1,\; \mu'_2 - (\mu_1')^2)^T$, where I have used standard notation for the raw moments, $E(x) = \mu'_1,\; E(x^2)=\mu'_2$, and $\sigma^2 =\mu'_2 - (\mu_1')^2$. The Fisher Information for this $\theta$ is

$$\mathcal I_{\theta}(\theta)=\begin{bmatrix} \frac {1}{\mu'_2 - (\mu_1')^2} & 0 \\ 0 & (1/2)\cdot\left(\frac {1}{\mu'_2 - (\mu_1')^2}\right)^2 \end{bmatrix}$$

What you want is the Fisher Information for the column vector $\eta= (\mu'_1,\; \mu'_2)^T$.
Note that $\theta$ can be expressed as a continuous function of $\eta$, $$\theta(\eta) = \left(0,\; - (\mu_1')^2\right)^T+ \eta$$

and so

$$\frac{\partial \theta(\eta)}{\partial\eta^T} = \begin{bmatrix} 1 & 0 \\ -2\mu'_1 & 1 \end{bmatrix}$$

Then apply chain-differentiation:

$$ \left(\frac{\partial}{\partial\eta} \log f(X;\theta(\eta))\right) = \left(\frac{\partial\theta(\eta)}{\partial\eta^T} \right)^T\cdot\left(\frac{\partial}{\partial\theta} \log f(X;\theta(\eta)\right)$$

taking the expected value of the outer product of the above, the Fisher Information of $\eta$ can be written

$$\mathcal{I}_{\eta}(\eta)=\left(\frac{\partial\theta(\eta)}{\partial\eta^T} \right)^T\cdot \operatorname{E} \left[ \left(\frac{\partial}{\partial\theta} \log f(X;\theta(\eta))\right)\cdot\left(\frac{\partial}{\partial\theta} \log f(X;\theta(\eta))\right)^T \right]\cdot\left(\frac{\partial\theta(\eta)}{\partial\eta^T} \right)$$

$$=\left(\frac{\partial\theta(\eta)}{\partial\eta^T} \right)^T\mathcal{I}_{\theta}(\theta)\left(\frac{\partial\theta(\eta)}{\partial\eta^T}\right)$$

I believe this is simpler to compute.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.