20
$\begingroup$

Yesterday's question Determine accuracy of model which estimates probability of event got me curious about probability scoring.

The Brier score $$\frac{1}{N}\sum\limits _{i=1}^{N}(\text{prediction}_i - \text{reference}_i)^2$$ is a mean squared error measure. Does the analogous mean absolute error performance measure
$$\frac{1}{N}\sum\limits _{i=1}^{N}|\text{prediction}_i - \text{reference}_i|$$ have a name, too?

Improve this question
$\endgroup$
4
  • $\begingroup$ Google allowed me to find this paper where something very similar is named $L_1$-calibration score. Note that this score is a bit different than yours, anyway "$L_1$ score" seems the good keyword. $\endgroup$
    – Elvis
    Commented Jan 4, 2012 at 17:23
  • $\begingroup$ What search terms did you use? Googling I mainly learned how many different tumour scores exist (L1 meaning lymphnode involvement in that context)... $\endgroup$
    – cbeleites
    Commented Jan 5, 2012 at 8:12
  • $\begingroup$ Something like "L1 score probability"... may be I've been lucky $\endgroup$
    – Elvis
    Commented Jan 5, 2012 at 8:17
  • 1
    $\begingroup$ Or google tries to help me and thinks I'm looking for tumours because that's what I do more often... "probability near score L1" got me to the paper below. $\endgroup$
    – cbeleites
    Commented Jan 5, 2012 at 8:24

1 Answer 1

Reset to default
24
$\begingroup$

Answer seems to be: no, because MAE doesn't lead to a proper scoring rule.

See Loss Functions for Binary Class Probability Estimation and Classification: Structure and Applications where the MAE is discussed under "Counterexamples of proper scoring rules".

Improve this answer
$\endgroup$
1
  • $\begingroup$ @amoeba: There you are $\endgroup$
    – cbeleites
    Commented Mar 11, 2014 at 17:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.