Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 3 down vote favorite

Yesterday's question Determine accuracy of model which estimates probability of event got me curious about probability scoring.

The Brier score $$\frac{1}{N}\sum\limits _{i=1}^{N}(prediction_i - reference_i)^2$$ is a mean squared error measure. Does the analogous mean absolute error performance measure
$$\frac{1}{N}\sum\limits _{i=1}^{N}|prediction_i - reference_i|$$ have a name, too?

share|improve this question
Google allowed me to find this paper where something very similar is named $L_1$-calibration score. Note that this score is a bit different than yours, anyway "$L_1$ score" seems the good keyword. – Elvis Jan 4 '12 at 17:23
What search terms did you use? Googling I mainly learned how many different tumour scores exist (L1 meaning lymphnode involvement in that context)... – cbeleites Jan 5 '12 at 8:12
Something like "L1 score probability"... may be I've been lucky – Elvis Jan 5 '12 at 8:17
1  
Or google tries to help me and thinks I'm looking for tumours because that's what I do more often... "probability near score L1" got me to the paper below. – cbeleites Jan 5 '12 at 8:24

1 Answer

up vote 4 down vote

Answer seems to be: no, because MAE doesn't lead to a proper scoring rule.

See Loss Functions for Binary Class Probability Estimation and Classification: Structure and Applications where the MAE is discussed under "Counterexamples of proper scoring rules".

share|improve this answer
This looks interesting, thanks for sharing – Elvis Jan 5 '12 at 9:28

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.