# What is the intuition behind the score function? [duplicate]

Wikipedia tells us that the score plays an important role in the Cramér–Rao inequality. It also phrases out the definition:

$$V = \frac{\partial}{\partial \theta} \log{L(\theta; X)}$$

However, I cannot find an intuitive explanation of what this quantity expresses. Obviously, it somehow measures how a small change of $\theta$ will affect the log-likelihood of the observed data $X$, but what exactly does that mean?

The wikipedia article also mentions that the expected value $\mathbb{E} [V \mid \theta] = 0$. Can this be interpreted somehow?

Going a bit further, in class we were told that the Fisher information (for which I have no intuitive understanding either) is $I(\theta) = \mathbb{E} [V^2 \mid \theta]$. Combined with $\mathbb{E} [V \mid \theta] = 0$ that would imply $I(\theta) = \text{Var}[V]$, is this correct?

PS: This is not homework.

• In regards to your last point, yes, Fisher information is the variance of the score. Can't help with intuition, though. – onestop Nov 24 '11 at 14:09
• As far as intuition of the score, do you understand the intuition of a derivative? – user5594 Nov 24 '11 at 17:42
• Strictly your final line should have been $I(\theta) = \text{Var}[V|\theta]$ – Henry Nov 25 '11 at 7:35
• @MikeWierzbicki: Yes, of course. But I was hoping there was more to it... – blubb Nov 25 '11 at 13:33
• For some intuition see stats.stackexchange.com/questions/196576/… – kjetil b halvorsen Nov 13 '19 at 0:39

The Wikipedia article gives an example of a Bernoulli process, with $A$ successes and $B$ failures and the probability of success $\theta$, where the score is $V = \frac{A}{\theta}-\frac{B}{1-\theta}$. If $\theta= \frac{A}{A+B}$, i.e. $\frac{\theta}{1-\theta}= \frac{A}{B}$, then $V=0$.
The score is more positive when there are more successes than would have been expected from the value of $\theta$, and more negative when there are fewer successes.