4
$\begingroup$

I have data in which groups of experts make proportion estimates.

I've been encouraged to use the ALRE method of scoring the error of these estimates. I found an article which describes this method:

enter image description here

As I understand it, the purpose of the ALRE method is to obtain a measure of error that is not dominated by single predictions that are wildly wrong. For example, answers to a question like "What is the distance (in km) between the Mars and Neptune?" could be wildly wrong.

However, all my data is bounded by 0 and 1 and thus I'm not sure if this measure is appropriate for me. If this is not a suitable measure for my sort of data, what should I use?

The observed values are 1s and 0s in some of my data sets. In some of my other data sets the observed values are also proportions.

$\endgroup$
2
  • 1
    $\begingroup$ Are the observed values that you have 1s or 0s? Or are they proportions too? I don't know this ALRE but if you have a predicted probability distribution, any proper scoring rule should work fine. See e.g. my answer here stats.stackexchange.com/questions/71720/… $\endgroup$
    – Momo
    Commented Jan 10, 2015 at 0:24
  • $\begingroup$ The observed values are 1s and 0s in some of my data sets. In some of my other data sets the observed values are also proportions. This is certainly something I should have mentioned in the OP, so I have edited it. Could you explain a little further what you mean by any proper scoring rule being fine if I have a predicted probability distribution? $\endgroup$ Commented Jan 10, 2015 at 0:34

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.