I need to estimate the probability that an outcome R comes from either the sum (SUM) or from the difference (DIF) of two independent random variables. To do that, I need to compute the conditional probabilities P(SUM|R) and P(DIF|R).

Suppose that the elements of the sum of two independent random variables are given by SUM = { X + Y } and that the elements of the difference are given by DIF = { X - Y }. The pdfs for the events SUM and DIF are then fsum(r) and fdif(r).

Since an outcome from the final sample space, S = SUM U DIF, may comes from either the sets SUM and DIF, how could I derivate the pdf for the final sample space, fs(r)?

Intuitively, I would use

$$ f_S(r) = Pr(SUM)f_{sum}(r) + Pr(DIF)f_{dif}(r) $$

where Pr(SUM) and Pr(DIF) are the probabilities of the events, but this clearly doesn't handle the joint density amount.

Supposing that R is an event from the final sample space S, we would have

$$ R = R \cap S = R \cap (S_{sum} \cup S_{dif}) = (R \cap S_{sum}) \cup (R \cap S_{dif}) $$


$$ F_R(r) = Pr( R \leq r ) = Pr\left [ (R \cap S_{sum}) \cup (R \cap S_{dif}) \leq r \right ] \\ \\ = Pr \left [ (R \cap S_{sum}) \leq r \right ] + Pr\left [ (R \cap S_{dif}) \leq r \right ] - Pr\left [ (R \cap S_{sum} \cap S_{dif}) \leq r \right ] \\ \\ = Pr \left [ (R | S_{sum}) \leq r \right ]Pr(S_{sum}) + Pr \left [ (R | S_{dif}) \leq r \right ]Pr(S_{dif}) - Pr \left [ (R | (S_{sum} \cap S_{dif})) \leq r \right ]Pr(S_{sum} \cap S_{dif}). $$

If I assume that the events SUM and DIF cannot happen at a same time and that they have the same probability of occurance, i.e. P(SUM) = P(DIF) = 1/2, how could I derivate fs(r) from the equations above?



$$ X \sim \mathcal{N}(u_x, \sigma_x^2), \\ Y \sim \mathcal{N}(u_y, \sigma_y^2), $$

and the probability of sum and difference are each $\frac{1}{2}$, then the resulting density is

$$ \frac{1}{2} \frac{1}{\sqrt{2 \pi (\sigma_x^2 + \sigma_y^2)}} e^{-\frac{\left(x - (u_x + u_y)\right)^2}{2(\sigma_x^2 + \sigma_y^2)}} + \frac{1}{2} \frac{1}{\sqrt{2 \pi (\sigma_x^2 + \sigma_y^2)}} e^{-\frac{\left(x - (u_x - u_y)\right)^2}{2(\sigma_x^2 + \sigma_y^2)}} $$

This follows by the law of total density probabilities, and the equations for sums (and differences) of independent normal variables. The law of total density probabilities is exactly what you wrote in "Intuitively, I would use". If a number of events partition the entire probability space, then the density is the sum of conditional densities each multiplied by the probability of the event.

It's also possible to ad-hoc derive this for this case as you continued doing, but the result will be the same. E.g., in your last

$$ Pr \left [ (R | S_{sum}) \leq r \right ]Pr(S_{sum}) + Pr \left [ (R | S_{sub}) \leq r \right ]Pr(S_{sub}) - Pr \left [ (R | (S_{sum} \cap S_{sub})) \leq r \right ]Pr(S_{sum} \cap S_{sub}). $$

you've correctly noted that the last term contains the intersection of mutually-exclusive events, and so the last term is 0. This is basicaly the law of total probabilities for cumulative densities.

As for the rest, you now need to differentiate by $r$ ($x$ in my equations) to get to the density. However, since

$$ Pr \left [ (R | S_{sum}) \leq r \right ] $$

is just the integral of the density, differentiating it will give the density.

  • $\begingroup$ If I differentiate $Pr[ (R|S_{sum}) \leq r]$, wouldn't I get the conditional pdf $f_{R|S_{sum}}$ instead of the marginal $f_{sum}(r)$? $\endgroup$
    – Rodrigo
    Jul 4 '17 at 12:57
  • 1
    $\begingroup$ @Rodrigo Let's put it in words a sec. It's differentiating, with respect to $r$, the probability that the result is less than $r$, assuming the result is the sum X the probability of sum. The result is just the PDF of the sum X $\frac{1}{2}$. $\endgroup$
    – Ami Tavory
    Jul 4 '17 at 13:02
  • $\begingroup$ I got it. Thank you. As a consequence, I could write the conditional probability $Pr(S_{sum}|R)$ as $\dfrac{Pr(R|S_{sum})Pr(S_{sum})}{Pr(R)}=\dfrac{Pr(S_{sum}){\int_{R}^{\:}f_{sum}(r)dr}}{ \int_{R}^{\:}{ \left[Pr(S_{sum})f_{sum}(r)+Pr(S_{dif})f_{dif}(r) \right]}dr}$, right? $\endgroup$
    – Rodrigo
    Jul 4 '17 at 14:00
  • $\begingroup$ @Rodrigo Yes, you can use Bayes to reverse the order indeed. $\endgroup$
    – Ami Tavory
    Jul 4 '17 at 14:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.