Reference for log-loss (cross-entropy)? I'm trying to track down the original reference for the logarithmic loss (logarithmic scoring rule, cross-entropy), usually defined as:
$$L_{log}=y_{true} \log(p) + (1-y_{true}) \log(1-p)$$
For the Brier score for example there is the Brier (1950) article. Is there such a reference for the log-loss?
 A: The earliest I have been able to find is

Good, I. J. “Rational Decisions.” Journal of the Royal Statistical Society. Series B (Methodological), vol. 14, no. 1, 1952, pp. 107–114. JSTOR, www.jstor.org/stable/2984087

Look at section 8, "Fair Fees":

By itself $\log p_1$ (or $\log(1 - p_1)$) is a measure of the merit of a probability estimate

I found this reference in Gneiting & Raftery (2007, JASA), who write that "This scoring rule dates back at least to Good (1952)", suggesting that they already did a similar search for original sources.
A: If we view minimizing cross entropy as equivalent to maximizing the log-likelihood of the same model, then I believe we can go as far back as RA Fisher. This places the date between 1912 and 1922, depending on how well-developed you wish the theory to be; see discussion in
John Aldrich "R. A. Fisher and the Making of Maximum
Likelihood 1912 – 1922" Statistical Science. 1997, Vol. 12, No. 3, 162-176
We also have some related threads:

*

*How to construct a cross-entropy loss for general regression targets?


*Definition and origin of “cross entropy”
which use the term "cross entropy" in the broad sense of a family of probabilistic losses, instead of the sense used in this post, as jargon for a specific loss for a model of binary data.
