17
$\begingroup$

Yesterday's question Determine accuracy of model which estimates probability of event got me curious about probability scoring.

The Brier score $$\frac{1}{N}\sum\limits _{i=1}^{N}(\text{prediction}_i - \text{reference}_i)^2$$ is a mean squared error measure. Does the analogous mean absolute error performance measure
$$\frac{1}{N}\sum\limits _{i=1}^{N}|\text{prediction}_i - \text{reference}_i|$$ have a name, too?

$\endgroup$
4
  • $\begingroup$ Google allowed me to find this paper where something very similar is named $L_1$-calibration score. Note that this score is a bit different than yours, anyway "$L_1$ score" seems the good keyword. $\endgroup$
    – Elvis
    Jan 4, 2012 at 17:23
  • $\begingroup$ What search terms did you use? Googling I mainly learned how many different tumour scores exist (L1 meaning lymphnode involvement in that context)... $\endgroup$ Jan 5, 2012 at 8:12
  • $\begingroup$ Something like "L1 score probability"... may be I've been lucky $\endgroup$
    – Elvis
    Jan 5, 2012 at 8:17
  • 1
    $\begingroup$ Or google tries to help me and thinks I'm looking for tumours because that's what I do more often... "probability near score L1" got me to the paper below. $\endgroup$ Jan 5, 2012 at 8:24

1 Answer 1

21
$\begingroup$

Answer seems to be: no, because MAE doesn't lead to a proper scoring rule.

See Loss Functions for Binary Class Probability Estimation and Classification: Structure and Applications where the MAE is discussed under "Counterexamples of proper scoring rules".

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.