Statistics : Why is the Cramer-Rao Lower Bound (CRLB) inverse of the Fisher Information I(θ) ? Why is the Cramer-Rao Lower Bound (CRLB) inverse of the Fisher Information I(θ) ? Could someone provide an intuitive explanation? I am having trouble understanding the concept.  
 A: I think this video gives a neat intuition, as it discusses the Cramer Rao Bound and Fisher information in a simple case in which geometric intuitions still work.
https://www.youtube.com/watch?v=i0JiSddCXMM
A: Almost two years later comes the longer answer: This is not a rigorous explanation but hopefully gives some intuition that the variance of the ML-estimator increases with the curvature of the log-likelihood (at least in the following simple example).
Assume that we have $m$ samples of size $n$ from $N(0, \sigma_1^2)$ and from $N(0, \sigma_2^2)$ where I pick $\sigma_1^2 = 1$ and $\sigma_2^2 = 2$.
The following graphs depict the log-likelihoods for these samples for $n = 20$ and $m=10$ (assuming we know the variance). The left side shows the samples for $\sigma_1^2$ and the right graph for
$\sigma_2^2$

Now the ML-estimator for one of these samples is the argmax of the respective function.
We can observe that:

*

*The ML-estimators on the left side have less variance than the ML-estimators on the right side

*The curvature of the graphs on the left side is considerable higher than the curvature on the right side.

So it seems to be that the curvature of the log-likelihood is inversely proportional to the variability of the ML-estimator.
So how can we make sense of this on an intuitive level?
If we consider the unimodal shape of the log-likelihood (which eventuall happens for most distributions once we have enough samples) the curvature of the log-likelihood gives an indication of how far away we can move from the best explanation of our current sample (i.e. the ML-estimate) and still get an almost as good as explanation (meaning a log-likelihood not much different). When we can move far away from our ML-estimate and still get a similarly good explanation, we should not be surprised to find out that in our next sample the best explanation is a very different one as in our current sample. If the curvature is very high and changes in paramters lead to a greater degree of explaindness (meaning log-likelihood) we would expect the best-guesses in other samples to be rather close to the best explanation for the current situation.
A: There is a certain correspondence between the variance of the estimator and the variance of the score or derivative of the likelihood. This becomes possibly more clear when we slightly rewrite the expression for the Cramer Rao bound instead of $\text{var}( \hat{\theta} ) \geq \frac{1}{I(\theta)}$ we can write it also as
$$\text{var}\left( \hat{\theta} \right) \cdot I(\theta) = \text{var}\left( \hat{\theta} \right) \cdot \text{var}\left( \frac{f^\prime({\bf x}; \theta)}{f({\bf x}; \theta)} \right)   \geq 1 $$
where we use the notation with a prime to denote a derivative with $\theta$, ie. $f^\prime(x,\theta) = \partial f(x,\theta)/\partial \theta$
from this expression it might become clear where from this inverse gets into the equation. We can derive it as a limit for the product of the estimator variance and the score variance.
The background of the above inequality is more precisely related to how the change in the expectation value of the estimator relates to the variance of the estimator and the Fisher information matrix
$$\text{var}\left( \hat{\theta}({\bf x}) \right) \cdot \text{var}\left( \frac{f^\prime({\bf x}; \theta)}{f({\bf x}; \theta)} \right) \geq \left(\text{cov}\left(\hat\theta({\bf x}) , \frac{f^\prime({\bf x}; \theta)}{f({\bf x}; \theta)} \right)\right)^2 = \left(\frac{\partial E[\hat{\theta}]}{\partial\theta}\right)^2\underbrace{  = 1  \vphantom{\frac{\partial E[\hat{\theta}]}{\partial\theta}}}_{\substack{\llap{\text{for unbiased estima}} \rlap{\text{tors we have $E[\hat{\theta}] = \theta$}} \\ \llap{\text{and the deriv}} \rlap{\text{ative equals 1}}}  }$$
This interaction between the $\hat{\theta}({\bf x})$ and the score $\frac{f^\prime({\bf x}; \theta)}{f({\bf x}; \theta)}$ can be seen as analogous to a torque in physics. The change of the estimator $\frac{\partial E[\hat{\theta}]}{\partial\theta}$ relates to the change of the density function (the force in the torque analogy) and the place where this change occurs (the distance/arm in the torque analogy).

