# What is the intuition behind the score function?

Wikipedia tells us that the score plays an important role in the Cramér–Rao inequality. It also phrases out the definition:

$$V = \frac{\partial}{\partial \theta} \log{L(\theta; X)}$$

However, I cannot find an intuitive explanation of what this quantity expresses. Obviously, it somehow measures how a small change of $\theta$ will affect the log-likelihood of the observed data $X$, but what exactly does that mean?

The wikipedia article also mentions that the expected value $\mathbb{E} [V \mid \theta] = 0$. Can this be interpreted somehow?

Going a bit further, in class we were told that the Fisher information (for which I have no intuitive understanding either) is $I(\theta) = \mathbb{E} [V^2 \mid \theta]$. Combined with $\mathbb{E} [V \mid \theta] = 0$ that would imply $I(\theta) = \text{Var}[V]$, is this correct?

PS: This is not homework.

-
In regards to your last point, yes, Fisher information is the variance of the score. Can't help with intuition, though. –  onestop Nov 24 '11 at 14:09
As far as intuition of the score, do you understand the intuition of a derivative? –  Mike Wierzbicki Nov 24 '11 at 17:42
Strictly your final line should have been $I(\theta) = \text{Var}[V|\theta]$ –  Henry Nov 25 '11 at 7:35
@MikeWierzbicki: Yes, of course. But I was hoping there was more to it... –  blubb Nov 25 '11 at 13:33
The Wikipedia article gives an example of a Bernoulli process, with $A$ successes and $B$ failures and the probability of success $\theta$, where the score is $V = \frac{A}{\theta}-\frac{B}{1-\theta}$. If $\theta= \frac{A}{A+B}$, i.e. $\frac{\theta}{1-\theta}= \frac{A}{B}$, then $V=0$.
The score is more positive when there are more successes than would have been expected from the value of $\theta$, and more negative when there are fewer successes.