11
$\begingroup$

My question is the following. Let's say I have two probability distributions:

$f(x|b), g(x|c)$

$b$ and $c$ are discrete events while $x$ is a continuous variable, i.e., when the button b is pressed there is some distribution for the amount of rain fall the next day, $x$.

When the button $c$ is pressed there is a different distribution of rain fall the next day, $x$. Are there any strategies for estimating the distribution of rain fall if both buttons are pressed, i.e.,

$h(x|b,c)$ ?

And, what assumptions do those strategies rest on?

$\endgroup$
6
  • $\begingroup$ You call both of your random variables (i.e., your events) x, but it might make more sense to call one y. Then we can ask, are x and y independent? If not, do you know how x and y are related? Without knowing the answers, it is not possible to get the joint probability distribution. That is, it is impossible to answer your question. $\endgroup$
    – Joel W.
    May 30, 2012 at 20:46
  • $\begingroup$ Can you say a and b are independant?i.e. p(a)*p(b)=p(a and b) $\endgroup$
    – Seth
    May 30, 2012 at 21:40
  • $\begingroup$ Seth - yes I can assume independence of b & c (I think you mean b and c as referred to in the question). $\endgroup$ May 30, 2012 at 21:50
  • $\begingroup$ Joel W. - well, the reason I only use x is that they are the same random variable. $\endgroup$ May 30, 2012 at 21:52
  • $\begingroup$ I don't see how this question makes sense. All you know is that b and c give different distributions for x. You don't even know what those distributions are. It could be that if b occurs you always get f(x|b) even when c occurs. Or what if c dominates then even if b occurs you get g(x|c). Those would be dependent cases. What would independence of b and c tell you? $\endgroup$ May 31, 2012 at 1:02

3 Answers 3

22
$\begingroup$

Short answer

The conditional distribution $P\left(x|yz\right)$ can be expressed in terms of $P\left(x|y\right)$ and $P\left(x|z\right)$ as $$ P\left(x|yz\right)\propto\frac{P\left(x|y\right)P\left(x|z\right)}{P\left(x\right)}, $$ where $P\left(x\right)$ is the prior on $x$ and $y$ and $z$ are conditionally independent given $x$ (see below for more details).

Long answer

Suppose we know the conditional distributions $P\left(x|y\right)$ and $P\left(x|z\right)$ which we would like to combine to obtain the distribution $P\left(x|yz\right)$. Using Bayes' theorem, we find $$ P\left(x|yz\right)=\frac{P\left(yz|x\right)P\left(x\right)}{P\left(yz\right)}. $$ We assume conditional independence of $y$ and $z$ given $x$ to obtain $$ \begin{align} P\left(x|yz\right)&=\frac{P\left(y|x\right)P\left(x\right)P\left(z|x\right)P\left(x\right)}{P\left(x\right)P\left(yz\right)}\\ &=\frac{P\left(y\right)P\left(z\right)}{P\left(yz\right)}\frac{P\left(x|y\right)P\left(x|z\right)}{P\left(x\right)}\\ &\propto\frac{P\left(x|y\right)P\left(x|z\right)}{P\left(x\right)}, \end{align} $$ where we have dropped the first term because it is only an overall normalisation.

Note: The above relation only holds if $y$ and $z$ are conditionally independent given $x$. Intuitively, this is the case if $y$ and $z$ are independent sources of information (see below for an example).

Example

Let $x=1$ if a sportsman took a performance enhancing drug, let $y=1$ if a drug test was positive, and let $z=1$ if the sportsman won a competition. The conditional independence assumption holds because the outcome of the drug test will not affect the outcome of the competition given $x$. Note that $y$ and $z$ are not unconditionally independent because the events are coupled by cheating.

Our prior suspicion of doping is $P\left(x\right)=\left(\begin{array}{cc}0.99 & 0.01\end{array}\right)$, where the first element corresponds to $x=0$ and the second corresponds to $x=1$. We assume that the test is 95% reliable such that $$ P\left(y|x\right)=\left(\begin{array}{cc} 0.95 & 0.05\\ 0.05 & 0.95 \end{array}\right), $$ where $y$ is the row index and $x$ is the column index. Furthermore, assume that a competitor gains a 5% advantage to win a competition by taking a performance enhancing drug such that $$ P\left(z|x\right)=\left(\begin{array}{cc} 1-p & 1-1.05\times p\\ p & 1.05\times p \end{array}\right), $$ where $p=0.1$ is the probability to win a competition if the sportsman has not taken a drug.

Using Bayes' theorem and the relation derived above, the conditional probabilities that the sportsman cheated are $$\begin{align} P\left(x=1|y\right) &=\left(\begin{array}{cc} 0.161017 & 0.000531\end{array}\right),\\ P\left(x=1|z\right) &=\left(\begin{array}{cc} 0.009995 & 0.010498\end{array}\right),\\ P\left(x=1|yz\right) &=\left(\begin{array}{cc} 0.000531 & 0.000558\\ 0.160949 & 0.167718 \end{array}\right), \end{align} $$ where $y$ is the row index and $z$ is the column index in the last equation. As expected, the drug test provides stronger evidence for cheating than winning a competition $P\left(x=1|y=1\right)>P\left(x=1|z=1\right)$ but both pieces of evidence provide an even stronger case for the sportsman cheating $P\left(x=1|y=1\cap z=1\right)>P\left(x=1|y=1\right)$.

$\endgroup$
2
  • $\begingroup$ Is $\frac{P\left(y\right)P\left(z\right)}{P\left(yz\right)}$ a constant for different observations of $y$ and $z$ respectively? Why can one drop it? $\endgroup$
    – psie
    Dec 6, 2020 at 22:19
  • 2
    $\begingroup$ We are primarily concerned with $x$, i.e. whether a performance-enhancing drug was taken. In other words, any term that does not depend on $x$ can be absorbed by the normalisation constant of the posterior for $x$. $\endgroup$ Dec 7, 2020 at 9:19
1
$\begingroup$

The solution is indeterminant. Even using p(b and c)= p(b) p(c) all we have is that the conditional density h(x|b and c) = h(x and b and c)/p(b and c)= h(x and b and c)/[p(b) p(c)]=h(x and c|b)/p(c). But this does nothing to relate the distribution h(x and c|b) to f(x|b) and g(x|c)

$\endgroup$
2
  • 1
    $\begingroup$ Thank you Michael. All I have found are strategies to combine pdf's in risk analysis, i.e. 10 'experts' forecast some pdf of risk (or an event related to risk) - how do you combine these to make a decision? There are a few strategies but it does not seem that any are derived from probability equations. $\endgroup$ May 31, 2012 at 1:36
  • $\begingroup$ @BrainPermafrost Would you mind listing a few links to the best info you found discussing these strategies? Thanks. $\endgroup$
    – walrus
    Oct 24, 2015 at 8:22
0
$\begingroup$

"10 'experts' forecast some pdf of risk (or an event related to risk) - how do you combine these to make a decision?

Assuming the experts come up with their pdfs using independent pieces of information, the unique correct way to combine the evidence is using the pointwise product of the density functions, just as we do when doing Bayesian estimation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.