# Fisher information always 0? What's wrong with this argument?

We exchange derivative and integral while proving that the expected value of the score is 0. Here is the proof which does so: \begin{align*} \int \left( \frac{\partial}{\partial \theta} \log p(x;\theta) \right) p(x;\theta) \,dx &= \int \frac{1}{p(x;\theta)} \frac{\partial}{\partial \theta} p(x;\theta) \,p(x;\theta)\,dx \\ &= \int \frac{\partial}{\partial \theta} p(x;\theta)\,dx \\ &= \frac{\partial}{\partial \theta} \int p(x;\theta)\,dx \\ &= \frac{\partial}{\partial \theta} 1 = 0 \end{align*} So $E_{\theta}\left[\frac{\partial}{\partial \theta} \log p(x;\theta)\right] = 0$. What's wrong with the following argument? \begin{align*} & E_{\theta}\left[ \frac{\partial}{\partial \theta} \log p(x;\theta) \right] = 0 \\ \Rightarrow & \frac{\partial}{\partial \theta}E_{\theta}\left[ \frac{\partial}{\partial \theta} \log p(x;\theta) \right] = 0 \\ \Rightarrow & E_{\theta} \left[ \frac{\partial}{\partial \theta} \frac{\partial}{\partial \theta} \log p(x;\theta)\right] = 0 \\ \Rightarrow & E_{\theta} \left[ \frac{\partial^2}{\partial \theta^2} \log p(x;\theta)\right] = 0 \\ \Rightarrow &\mathcal{I}(\theta) = 0 \end{align*}

• If you write out the expectation again as an integral you will see that you haven't actually interchanged integration and differentiation.
– whuber
Commented Nov 22, 2014 at 15:33
• Ah, I see your point. There's also the $p(x;\theta)$ density term. Thanks whuber! Commented Nov 22, 2014 at 15:37

Starting from your third line:

$0 = E_\theta\left(\frac{d}{d\theta}\frac{d}{d\theta}\log(p(x;\theta))\right) = \int \frac{d}{d\theta}\left(\frac{d}{d\theta}\log(p(x;\theta))p(x;\theta)\right)dx$

$= \int \left[\frac{d^2}{d\theta^2}\log(p(x;\theta))p(x;\theta)+\frac{d}{d\theta}p(x;\theta)\frac{d}{d\theta}\log(p(x;\theta))\right]dx$

$= \int \left[\frac{d^2}{d\theta^2}\log(p(x;\theta))p(x;\theta)+\frac{d}{d\theta}\log(p(x;\theta))p(x;\theta)\frac{d}{d\theta}\log(p(x;\theta))\right]dx$

$\implies 0 = E\left(\frac{d^2}{d\theta^2}\log(p(x;\theta))\right) + E\left((\frac{d}{d\theta}\log(p(x;\theta)))^2\right)$

$\implies -E\left(\frac{d^2}{d\theta^2}\log(p(x;\theta))\right) = E\left((\frac{d}{d\theta}\log(p(x;\theta)))^2\right) = I(\theta)$

The problem is that from your third to fourth line the derivative is applied to the entire expression in the expectation. So you need to use the product rule on the term $\frac{d}{d\theta}\log(p(x;\theta))p(x;\theta)$.