Let $\mathbf{X} = (X_1, \ldots, X_n)$ be an i.i.d sample from the parametric family of distributions $\mathcal{P} = \{P_\theta: \theta \in \Theta \}$ ($X_i \sim P_{\theta_0}$ are i.i.d. random variables, $\theta_0 \in \Theta$ is a fixed true parameter).
I am studying statistics and trying to understand the following formula for Fisher information:
$$I(\theta) = \mathrm{E}_\theta\left[ \left(\frac{\partial \log f(X_1; \theta)}{\partial \theta} \right)^2 \right]$$
I see the following algorithm for computing $I(\theta)$:
- Using that $X_1 \sim P_{\theta_0}$ we compute the quantity $g(X_1; \theta) = \left(\frac{\partial \log f(X_1; \theta)}{\partial \theta} \right)^2$. Since $\theta_0$ is fixed, we can say that $X_1$ doesn't depend on $\theta$ and the derivative $\frac{\partial}{\partial \theta}$ is easily computed (at this stage we can even treat $X_1$ as some fixed constant).
- After that we need to assume that $X_1 \sim P_{\theta}$ and compute the expected value $\mathrm{E}_\theta[g(X_1; \theta)]$.
My question is the following: is this algorithm correct?
I am concerned about the non-trivial fact of "changing" $X_1$ distribution from $P_{\theta_0}$ to $P_{\theta}$ at the second stage.