9
$\begingroup$

The score of a random variable is the partial derivative w.r.t. $\theta$ of the log likelihood of that parameter given a dataset $x$.

I'm having trouble interpreting this quantity. Why it is called "score"?

$\endgroup$
2
  • 3
    $\begingroup$ How fast the log likelihood is changing with possible parameter values, and we most often want to find where that log likelihood is maximum. If there is a story about why "score" I would like to hear it too. My impression: It's just one of various words that can be used and I don't know that it ever has a unique meaning. Thus I hear of test scores (marks, grades), scores in various sports, standard or $z$ score; this sense; and there may well be other senses. $\endgroup$
    – Nick Cox
    Jan 31, 2018 at 18:41
  • $\begingroup$ I don't know the answer for the ubiquity of the term "score." Perhaps this comes from measurement theory? $\endgroup$ Jan 31, 2018 at 19:51

1 Answer 1

11
$\begingroup$

This one is an interesting historical curiosity. The term "score" was initially used in Fisher (1935) in a statistical problem involving the analysis of genetic properties of families with a parent that has a genetic abnormality. Fisher noted that the children of an abnormal parent can be classified into four classes, being the combinations of their binary inheritance status (inheritance or non-inheritance of the abnormality), and their binary zygosity (homozygous or heterozygous).

Fisher proposed that a family observed in the data could be given a "score" based on the number of children in each of these four categories, and this score could be used to estimate the linkage parameter describing the inheritance probability of interest in his problem. He measured the efficiency of his proposed score, and an alternative scoring rule by another author, by comparing both to the "ideal score" obtained by assigning each data point a score equal to the derivative of the logarithm of the sampling density (p. 193).

In later work, other authors followed Fisher's method of analysis but applied it to more general contexts where they were no longer trying to allocate a "score" to a family in a genetic problem. In this later work, authors still speak of the "score" or "efficient score", but they use this in a more generalised sense, as a direct referent to the derivative of the log-likelihood. This progression culminated in a paper by Rao (1948) where the author introduces "efficient score tests" using the derivative of the log-likelihood function.

So, the term "score" initially arose as a term that Fisher used in a specific application of statistics to a problem involving the genetic properties of children in a family. He estimated a genetic probability by giving each family a "score" based on the number of children in four categories of interest. The term was then deployed more broadly by other authors to refer to what Fisher had called the "ideal score", which was the derivative of the log-likelihood.


Note on reading Fisher's paper: This paper is a bit hard to read, since it uses statistical inference methods but describes them in an antiquated way. Fisher makes reference to statistical inference methods and information theory, but does not give references for his steps. Essentially what he is saying is the following.

Suppose we let $f(x| \xi)$ be the density for the observed values, and let $\ell_{x}(\xi) \equiv \ln f(x | \xi)$ be the corresponding log-density, with parameter $\xi$. The "ideal score" assigned to a single data point (e.g., a count of children in a genetic category) is:

$$\text{Ideal score for }x_i = \frac{1}{f(x_i | \xi)} \cdot \frac{df}{d \xi}(x_i | \xi) = \frac{d\ell_{x_i}}{d \xi} (\xi).$$

The ideal score for the entire data set (e.g., counts of a family of children in several genetic categories) is obtained by adding up the individual scores for the data points, yielding:

$$\text{Ideal score for data set }\boldsymbol{x} = \sum_{i=1}^k \frac{d\ell_{x_i}}{d \xi} (\xi) = \frac{d\ell_\boldsymbol{x}}{d \xi}(\xi).$$

As you can see, this is just the derivative of the log-likelihood function for the entire data set (what we now refer to as the "score function"). Fisher uses the term "score" in the generic sense of a quantity applied to a unit (a family) to rank those units on a scale for his purposes. He was clearly able to see that the derivative of the log-likelihood represented an "ideal" of this "score" for the purposes of his work.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.