4
$\begingroup$

I have data in which groups of experts make proportion estimates.

I've been encouraged to use the ALRE method of scoring the error of these estimates. I found an article which describes this method:

enter image description here

As I understand it, the purpose of the ALRE method is to obtain a measure of error that is not dominated by single predictions that are wildly wrong. For example, answers to a question like "What is the distance (in km) between the Mars and Neptune?" could be wildly wrong.

However, all my data is bounded by 0 and 1 and thus I'm not sure if this measure is appropriate for me. If this is not a suitable measure for my sort of data, what should I use?

The observed values are 1s and 0s in some of my data sets. In some of my other data sets the observed values are also proportions.

$\endgroup$
  • 1
    $\begingroup$ Are the observed values that you have 1s or 0s? Or are they proportions too? I don't know this ALRE but if you have a predicted probability distribution, any proper scoring rule should work fine. See e.g. my answer here stats.stackexchange.com/questions/71720/… $\endgroup$ – Momo Jan 10 '15 at 0:24
  • $\begingroup$ The observed values are 1s and 0s in some of my data sets. In some of my other data sets the observed values are also proportions. This is certainly something I should have mentioned in the OP, so I have edited it. Could you explain a little further what you mean by any proper scoring rule being fine if I have a predicted probability distribution? $\endgroup$ – user1205901 - Reinstate Monica Jan 10 '15 at 0:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.