I have data in which groups of experts make proportion estimates.
I've been encouraged to use the ALRE method of scoring the error of these estimates. I found an article which describes this method:
As I understand it, the purpose of the ALRE method is to obtain a measure of error that is not dominated by single predictions that are wildly wrong. For example, answers to a question like "What is the distance (in km) between the Mars and Neptune?" could be wildly wrong.
However, all my data is bounded by 0 and 1 and thus I'm not sure if this measure is appropriate for me. If this is not a suitable measure for my sort of data, what should I use?
The observed values are 1s and 0s in some of my data sets. In some of my other data sets the observed values are also proportions.