2
$\begingroup$

Vose (in Risk analysis a quantitative guide, 2008) argues that it is preferable to use non-parametric distributions when eliciting knowledge about an unknown distribution from experts. The argument is that experts may not be familiar with e.g. the shape parameter of a Weibull distribution and would perform better if using other methods of describing a distribution that would then be non-parametric.

Let’s consider a triangular distribution. Let’s say I ask one expert about the lowest, most likely and highest value a stochastic variable can take and derive a simple triangular distribution from this. Next I ask another expert with slightly differing views on the three values, leading me to the question of how to aggregate / combine these estimates (assuming that both have real expertise and are unbiased / reasonably well calibrated). Would it be valid to assume that the expert’s estimates follow some stochastic distribution (e.g. normal, gamma) and can be combined through a Bayesian update? To stick to the closed form solution I would further assume conjugate priors for the distribution of the expert's estimates. In this update the first expert’s estimates would constitute the prior, the second expert’s would constitute the likelihood from which the posterior for each estimate (low, most likely, high) would be derived. Are there any established methods for this application?

$\endgroup$
  • 1
    $\begingroup$ The use of the term "nonparametric distribution" seems odd here. How is a triangular distribution anything but parametric? In what sense is it any less parametric than the Weibull? Certainly experts may be unfamiliar with how to interpret (or convey) information from some particular parameterization of some specific distribution, but things like "mean", "median" or "mode" -- or indeed "90th percentile" or "minimum" should be well understood and will work perfectly well with a variety of distributions (one can match moments or quantiles to get parameters just fine, for example). ... ctd $\endgroup$ – Glen_b -Reinstate Monica Apr 20 '17 at 13:53
  • $\begingroup$ ctd ... It's not the choice of the distribution that seems to be the source of the difficulty for experts being discussed, but the choice of how to get information about a distribution. [A "nonparametric distribution" would be one without a fixed (and potentially unbounded) number of parameters, such as you'd get with a kernel density estimate)]. Of course the issue of combining information from multiple experts would remain -- my concern is mainly with the initial description of the issue. $\endgroup$ – Glen_b -Reinstate Monica Apr 20 '17 at 14:02
  • $\begingroup$ Btw, there is no conjugate prior for parameters of triangular distribution. $\endgroup$ – Tim Apr 20 '17 at 15:01
  • $\begingroup$ @Glen_b: thanks for the comments! In labeling a triangular distribution non-parametric I followed the above mentioned source (which is an applied simulation source, not a "proper" stats textbook). However, I agree that a triangular distribution is defined by a fixed set of parameters thus making it parametric. $\endgroup$ – Daniel C Apr 20 '17 at 15:21
  • $\begingroup$ Yes, my terminological dispute would be with Vose... $\endgroup$ – Glen_b -Reinstate Monica Apr 20 '17 at 15:26
1
$\begingroup$

I don't really think that what you are describing is a Bayesian updating problem. If the expert' opinions are (or can be assumed) independent, then together they form a joint distribution

$$ g(x) = f_1(x) \times f_2(x) \times \dots \times f_k(x)$$

Triangular distribution has a fixed support, so the product of the distribution would zero-out the non-overlapping regions (this may be seen as a advantage, or a disadvantage depending on your aim). On another hand, you can combine together the opinions into a mixture distribution

$$ g(x) = \sum_i w_i f_i(x) $$

(a.k.a. linear opinion pool) where $w_i$ are the non-negative weights such that $\sum_i w_i = 1$, with $w_1 = w_2 = \dots = w_k = 1/k$ if you want to treat all the opinions as equally trustworthy. Yet another alternative is logarithmic opinion pool:

$$ g(x) = \frac{ \prod_i f_i(x)^{w_i} } { \int \, \prod_i f_i(x)^{w_i} \, d\mu }$$

For more examples and further details see

Clemen, R.T. and Winkler, R.L. (1999). Combining Probability Distributions From Experts in Risk Analysis. Risk Analysis, 19(2), 187-203.

Genest, C. and Zidek, J. V. (1986). Combining probability distributions: A critique and an annotated bibliography. Statistical Science, 114-135.

and Combining probabilities/information from different sources for discussion of combining individual probabilities.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.