4
$\begingroup$

I have a hidden binary random variable Z that can have a value of either 1 or 0. There is some true probability P(Z=1) = z that I do not know.

I also have two separate pieces of "evidence" that give some information about the state of Z:

  • P(Z=1) = x
  • P(Z=1) = y

I'm interested in estimating z by combining the evidence contained in x and y. In particular:

  • Is there a simple formula I can use to express a good estimate for z in terms of x and y?
  • If I "trust" one of the two pieces of evidence more than the other, is there a way of weighting the combination?

Both pieces of evidence can be viewed as imperfect estimates of the true probability z, from alternative sources that have access to some (but not all) information that might be useful for predicting the random variable Z (e.g they can observe some variables that are correlated with Z). E.g. you have two bookmakers who are independently giving you odds of a specific event occurring, but neither has perfect information so the odds they give you are different guesses at the true probability.

So the motivation for this question is whether, knowing two separate predictions, you can somehow combine them to make a single better prediction of z.

$\endgroup$
5
  • 1
    $\begingroup$ It seems to me this problem has nothing to do with a random variable $Z$. There is an unknown number $z$ and two pieces of "evidence" $x$ and $y$: that's it. This simple description makes it abundantly clear that your question cannot be answered until you explain how $x$ and $y$ were obtained. Did you consult the oracle of Apollo at Delphi? Ask two passing strangers on the street? Look up the results of two double-blinded controlled studies? Pick the two highest values out of a collection of 100 Web articles? :-) $\endgroup$
    – whuber
    Aug 27, 2011 at 14:59
  • $\begingroup$ Does it matter how the evidence was obtained? I've edited the question to make it a bit clearer. $\endgroup$
    – mikera
    Aug 27, 2011 at 15:16
  • $\begingroup$ Have you thought about how the answers could differ in the four scenarios I described? How the evidence was obtained is absolutely critical. (There's no record of any edits to the question, by the way.) $\endgroup$
    – whuber
    Aug 27, 2011 at 15:18
  • $\begingroup$ Not sure why you can't see the edits. Maybe you need to refresh? Hopefully they make the question clearer - though I'm not a professional statistician so apologies if I am not using the most appropriate terminology :-) $\endgroup$
    – mikera
    Aug 27, 2011 at 15:49
  • $\begingroup$ I gues you need to ask about average among models, since it is as if you have two models and they make different predictions and you want to average these predictions. There is a huge literature on the subject. Maybe someone can point you to a godd start. $\endgroup$ Aug 27, 2011 at 17:11

1 Answer 1

2
$\begingroup$

One way to think about this is to let each piece of "evidence" be represented by a Beta distribution, $B(a_i+1,b_i+1)$, where $a_i,b_i \ge 0$. Let $P_i(Z=1) = \int_0^1 x\,B(x|a_i+1,b_i+1)\,dx = (a_i+1)/(a_i+b_i+2)$. Associated with each piece of evidence is its precision, which is the inverse of its variance. The variance is $(a_i+1)(b_i+1)/[(a_i+b_i+2)^2 (a_i+b_i+3)]$.

Combining a number of pieces of evidence produces another Beta distribution, $B(c+1,d+1)$ where $c=\sum a_i$ and $d=\sum b_i$. (Assuming the pieces of evidence are independent, the probability density function (PDF) for the combined evidence equals the product of the individual PDFs, normalized to integrate to one.) Thus the combined evidence produces $P_{\text{comb}}(Z=1) = (c+1)/(c+d+2)$. The individual precisions are taken into account automatically: more precise evidence gets more weight. Finally, the precision associated with the combined result is the inverse of the variance of its Beta distribution, $(c+1)(d+1)/[(c+d+2)^2(c+d+3)]$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.