The Fisher information measures the curvature of the log-likelihood and can be used to assess the efficiency of estimators.

The Fisher information is the first derivative of the score function $u(\theta ;X)$ or it can be obtained as the expected value of the observed information $$I( \theta_0) = E_{ \theta_0} \big( \big( \frac{\partial}{\partial \theta_0} \ln{ f(X|\theta_0)|_{\theta = \theta_0}} \big)^2 \big). $$

When the second derivative of the log likelihood exists, under certain regularity conditions, it can also be written as:

$$I( \theta) = -E\big(\frac{\partial^2}{\partial \theta^2} \ln{ f(X;\theta) }|\theta \big)$$

It provides the expected curvature of the log-likelihood, hence greater values for $I(\theta_0)$ imply greater curvature and therefore more information about the true value of $\theta$. Higher curvature means that the likelihood function is more peaked and has smaller variance around the maximum. The Fisher information can be used to compare estimators in terms of their efficiency and is used in the Cramer-Rao inequality to provide a lower bound (which is the inverse of the Fisher information) on the variance of the unbiased estimator of $\theta$.

history | excerpt history