1
$\begingroup$

I have sampled a set of expert on their 90% CI for a set of questions. How can I combine their answers? How confident can I be in the result?

|---------------------------------------|
|     Question 1                        |   
|---------------------------------------|
|         | Lower bound  | Upper bound  |
|         | (95% chance  | (95% chance  |
|         |  value is    |  value is    |
| Expert  |  higher)     |  lower)      |
|---------|--------------|--------------|
|  A      | 130          |  627         |
|  B      | 202          |  584         |
|  C      | 186          |  648         |

Ideally, I'd like to understand what my lower and upper bounds are likely to be and how confident I should be in these bounds.

Improve this question
$\endgroup$
  • 1
    $\begingroup$ Confidence intervals do not tell you how confident you could be about the results, neither they tell you about the chance of value being higher or lower (see other questions tagged as confidence-interval, e.g. stats.stackexchange.com/q/26450/35989). What exactly do you want to achieve? $\endgroup$ – Tim♦ Jan 17 '18 at 18:23
  • $\begingroup$ Say, for example, I asked 5 astronomers what the diameter of the moon was (without looking it up). They may not know off the top of their heads, but they know it's almost certainly larger than X and smaller than Y. Can I take those 5 estimates and combine them to a single estimate that is superior than any of the 5 individually? $\endgroup$ – PaulH Jan 17 '18 at 18:54
  • $\begingroup$ You can't combine these. The Experts are giving Bayesian credible intervals based on their priors. The priors don't have any frequentist properties. They can't be averaged to produce meaningful results. It is a conflagration of qualitative and quantitative results with no actual data to update the priors. $\endgroup$ – AdamO Jan 17 '18 at 19:55
1
$\begingroup$

This might be easier if you commit to taking a Bayesian approach (which you pretty already have). There is a rich literature on methods to elicit and combine distributions for use as subjective priors. See a few examples linked below to get you started:

https://projecteuclid.org/download/pdfview_1/euclid.ba/1346158771

https://www.sciencedirect.com/science/article/pii/S1364815213002533

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.