Let $\mathbf{X} = (X_1, \ldots, X_n)$ be an i.i.d sample from the parametric family of distributions $\mathcal{P} = \{P_\theta: \theta \in \Theta \}$ ($X_i \sim P_{\theta_0}$ are i.i.d. random variables, $\theta_0 \in \Theta$ is a fixed true parameter).

I am studying statistics and trying to understand the following formula for Fisher information:

$$I(\theta) = \mathrm{E}_\theta\left[ \left(\frac{\partial \log f(X_1; \theta)}{\partial \theta} \right)^2 \right]$$

I see the following algorithm for computing $I(\theta)$:

  1. Using that $X_1 \sim P_{\theta_0}$ we compute the quantity $g(X_1; \theta) = \left(\frac{\partial \log f(X_1; \theta)}{\partial \theta} \right)^2$. Since $\theta_0$ is fixed, we can say that $X_1$ doesn't depend on $\theta$ and the derivative $\frac{\partial}{\partial \theta}$ is easily computed (at this stage we can even treat $X_1$ as some fixed constant).
  2. After that we need to assume that $X_1 \sim P_{\theta}$ and compute the expected value $\mathrm{E}_\theta[g(X_1; \theta)]$.

My question is the following: is this algorithm correct?

I am concerned about the non-trivial fact of "changing" $X_1$ distribution from $P_{\theta_0}$ to $P_{\theta}$ at the second stage.


1 Answer 1


I think I found the answer. There are 2 mistakes in the post above.

1) Our statistical model is $X_1 \sim P_\theta$ where $\theta \in \Theta$, not $X_1 \sim P_{\theta_0}$. We don't know the true value of the parameter. So there is no "change" of distribution.
2) $X_1(\omega)$ as function of outcomes $\omega$ doesn't depend on parameter $\theta$. Indeed, every random variable is just a measurable function that maps $\Omega \to \mathbb{R}$ (by definition). Only the distribution $P_\theta$ of $X_1(\omega)$ depends on $\theta$ but not $X_1(\omega)$ itself. Hence $\frac{d X_1(\omega)}{d \theta} = 0$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.