# The Fisher information matrix

I have read two definitions of the Fisher information matrix. Are they equivalent? (The formulas given below are slightly modified versions of those given in the indicated sources, so as to bring out the differences between them.)

1. Zacks, The Theory of Statistical Inference (1971), p. 194: $$\left[E_\theta\left(\frac{\partial}{\partial\theta_i}\ln f\left(X;\theta_1,\dots,\theta_r\right)\cdot\frac{\partial}{\partial\theta_j}\ln f\left(X;\theta_1,\dots,\theta_r\right)\right)\right]_{i,j}$$
2. Schervish, Theory of Statistics (1997), Definition 2.79, p. 111: $$\mathrm{Cov}_\theta\left(\frac{\partial}{\partial\theta_i}\ln f\left(X;\theta_1,\dots,\theta_r\right),\frac{\partial}{\partial\theta_j}\ln f\left(X;\theta_1,\dots,\theta_r\right)\right)$$
• That's the same as saying "when does $\text{Cov}(X,Y) = \text{E}(XY)$?" ... which is when $\text{E}(X)\text{E}(Y) = 0$ ... so you should think about the circumstances under which they're the same. That is, when does $\frac{\partial}{\partial\theta_i}\ln f\left(X;\theta_1,\dots,\theta_r\right) = 0$? Jun 13 '13 at 8:47
• @Glen_b: That's a good question! I would say that $\frac{\partial}{\partial\theta_i}\ln f\left(X;\theta_1,\dots,\theta_r\right)=0$ at a point of extremum of $f$ w.r.t. $\boldsymbol\theta$. Jun 13 '13 at 8:51
• @EvanAad note quite. Do not forget that you take the expected value over $X$. Jun 13 '13 at 9:31

Fortunately they are the same. All we need to prove is $E_{\theta} \left[ \frac{\partial}{\partial \theta} \log f(x|\theta) \right] = 0$, so here we go.

First note that

$$\int_{\mathbb{R}} \frac{\partial}{\partial \theta}\log(f(x|\theta)) \cdot f(x|\theta) dx = \int_{\mathbb{R}} \frac{\partial f(x|\theta)}{\partial \theta}dx.$$

Under the conditions of Leibniz's integral rule you can rewrite the above

$$\frac{\partial}{\partial \theta} \int_{\mathbb{R}} f(x|\theta)dx = \frac{\partial}{\partial \theta} 1 = 0.$$

So, except pathological cases where the conditions mentioned above do not hold, the expected value of the score is 0, and the covariance is the same as the expected value of the product.

• Thank you. Could you have meant Leibniz's integral rule rather than Lebesgue's dominated convergence theorem? Jun 13 '13 at 10:45
• Indeed! I had in mind point 3 of the measure theory statement of the rule, but of course point 1 and 2 are also required. I will edit my answer accordingly. Jun 13 '13 at 19:58