# The Fisher information matrix

I have read two definitions of the Fisher information matrix. Are they equivalent? (The formulas given below are slightly modified versions of those given in the indicated sources, so as to bring out the differences between them.)

1. Zacks, The Theory of Statistical Inference (1971), p. 194: $$\left[E_\theta\left(\frac{\partial}{\partial\theta_i}\ln f\left(X;\theta_1,\dots,\theta_r\right)\cdot\frac{\partial}{\partial\theta_j}\ln f\left(X;\theta_1,\dots,\theta_r\right)\right)\right]_{i,j}$$
2. Schervish, Theory of Statistics (1997), Definition 2.79, p. 111: $$\mathrm{Cov}_\theta\left(\frac{\partial}{\partial\theta_i}\ln f\left(X;\theta_1,\dots,\theta_r\right),\frac{\partial}{\partial\theta_j}\ln f\left(X;\theta_1,\dots,\theta_r\right)\right)$$
• That's the same as saying "when does $\text{Cov}(X,Y) = \text{E}(XY)$?" ... which is when $\text{E}(X)\text{E}(Y) = 0$ ... so you should think about the circumstances under which they're the same. That is, when does $\frac{\partial}{\partial\theta_i}\ln f\left(X;\theta_1,\dots,\theta_r\right) = 0$? Commented Jun 13, 2013 at 8:47
• @Glen_b: That's a good question! I would say that $\frac{\partial}{\partial\theta_i}\ln f\left(X;\theta_1,\dots,\theta_r\right)=0$ at a point of extremum of $f$ w.r.t. $\boldsymbol\theta$. Commented Jun 13, 2013 at 8:51
• @EvanAad note quite. Do not forget that you take the expected value over $X$. Commented Jun 13, 2013 at 9:31

Fortunately they are the same. All we need to prove is $E_{\theta} \left[ \frac{\partial}{\partial \theta} \log f(x|\theta) \right] = 0$, so here we go.

First note that

$$\int_{\mathbb{R}} \frac{\partial}{\partial \theta}\log(f(x|\theta)) \cdot f(x|\theta) dx = \int_{\mathbb{R}} \frac{\partial f(x|\theta)}{\partial \theta}dx.$$

Under the conditions of Leibniz's integral rule you can rewrite the above

$$\frac{\partial}{\partial \theta} \int_{\mathbb{R}} f(x|\theta)dx = \frac{\partial}{\partial \theta} 1 = 0.$$

So, except pathological cases where the conditions mentioned above do not hold, the expected value of the score is 0, and the covariance is the same as the expected value of the product.

• Thank you. Could you have meant Leibniz's integral rule rather than Lebesgue's dominated convergence theorem? Commented Jun 13, 2013 at 10:45
• Indeed! I had in mind point 3 of the measure theory statement of the rule, but of course point 1 and 2 are also required. I will edit my answer accordingly. Commented Jun 13, 2013 at 19:58