4
$\begingroup$

If $P(x \mid y)$ is high, for example, above 0.5, does it imply that $P(y \mid x)$ is also high (above $0.5$)? If yes/no, then why? What if I know that they are dependent on each other? For what type of events may this relation hold?

$\endgroup$
14
$\begingroup$

I like this example: $$ \DeclareMathOperator{\P}{\mathbb{P}} \P(\text{Dead} \mid \text{Was hanged}) \qquad \text{is high}\\ \P(\text{Was hanged} \mid \text{Dead}) \qquad \text{is low} $$ and that should answer your question.

$\endgroup$
3
  • 3
    $\begingroup$ @Ahmad see my edit, it is not about if they are dependent, but rather what is the nature of the dependence. Being dead is strictly dependent on being hanged, more than this, there is a causal relationship, otherwise, it would not be used as a form of capital punishment. $\endgroup$
    – Tim
    May 1 at 19:50
  • $\begingroup$ @Tim, Sure, right! Death could be due to many things. I meant when you hear one was hanged, it gives you a high probability or information that death has occurred, otherwise, it would not be used as a form of capital punishment. However, when you hear one was dead, it tells nothing of a few about whether it was hanged or not. But yes, I must use other terminolgy. $\endgroup$
    – Ahmad
    May 2 at 5:59
  • $\begingroup$ @Tim, I revised my answer once again and clarified what was my intubation. You may want to consider if it has a problem or not. $\endgroup$
    – Ahmad
    May 3 at 10:39
8
$\begingroup$

$P(x|y)$ and $P(y|x)$ are different things and do not have to be related. Say that you have an “algorithm” that predicts the weather. It is very simple, it always predicts “it is going to rain”. So P(predicted rain|rain) = 1, in all cases it rained, it also predicted rain. On another hand, say that in your area it rains only 5% of the time, so P(rain|predicted rain) = 0.05, since only in 5% of cases when it predicted rain (always), it rained.

Think of it geometrically. Say that you are throwing stones into a box split into four quadrants while being blindfolded (i.e. at random). If I tell you that you threw the stone into the quadrant within the $x=1$ column, the conditional probability that it was $y=1$ (you're still blindfolded!), $p(y=1|x=1)=1/2$ would be the area of $y=1,x=1$ quadrant, divided by the area of the $x=1$ column. For $p(x=1|y=1)=1/2$, the calculation would be done row-wise.

$$ \begin{array}{c|c|c|} & x=0 & x=1 \\ \hline y=0 & & \\ \hline y=1 & & \\ \hline \end{array} $$

Now imagine that you expand the area of the $y=1$ row twice. The marginal $p(y=1)=2/3$ and conditional probabilities $p(y=1|x=1)=2/3$ have changed. However, if you look at the relative areas from the $x$'s point of view, the relative areas $p(x=1|y=1)=1/2$ are still the same.

$$ \begin{array}{c|c|c|} & x=0 & x=1 \\ \hline y=0 & & \\ \hline y=1\\\phantom{y=1} & & \\ \hline \end{array} $$

The same can be done if you expanded the area of any of the columns instead, the result would be the opposite.

If you changed the areas on both of the dimensions, both the conditional probabilities would change. Play around with it (in your head, using pen and pencil, or using a computer) and check how do the relative areas change. If you expand the row together with the column, say making $x=1$ twice as big and $y=1$ twice as big, would make both conditional probabilities equal to $2/3$.

$$ \begin{array}{c|c|c|} & x=0 & x=1\phantom{x=1} \\ \hline y=0 & & \\ \hline y=1\\\phantom{y=1} & & \\ \hline \end{array} $$

But you could also do the scaling in an opposite way, expand $x=0$ and $y=1$, in such case $p(x=1|y=1)=1/3$ and $p(y=1|x=1)=2/3$.

$$ \begin{array}{c|c|c|} & x=0\phantom{x=0} & x=1 \\ \hline y=0 & & \\ \hline y=1\\\phantom{y=1} & & \\ \hline \end{array} $$

$\endgroup$
15
  • 1
    $\begingroup$ "So P(predicted rain|rain) = 1, in all cases it predicted rain, it rained." doesn't say clearly what I think you wanted to say. $\endgroup$
    – user20637
    May 1 at 18:30
  • $\begingroup$ Thank you, however, they are related through the Bayes rule. $\endgroup$
    – Ahmad
    May 1 at 18:58
  • $\begingroup$ @Ahmad Bayes theorem tells you only how to calculate one from another, it tells you nothing about them being related. $\endgroup$
    – Tim
    May 1 at 19:03
  • $\begingroup$ @Tim, thank you, what if I know that they are dependent? I add this to my original question. $\endgroup$
    – Ahmad
    May 1 at 19:06
  • 1
    $\begingroup$ @Ahmad what does their equal shared area tell you about the both conditional probabilities? It’s same case as in my answer if you think about it. $\endgroup$
    – Tim
    May 1 at 21:09
2
$\begingroup$

Let's review the conditional probability of both $P(x|y)$ and $P(y|x)$:

$$P(x|y) = \frac{P(x \cap y)}{P(y)}$$

and

$$P(y|x) = \frac{P(x \cap y)}{P(x)}$$

The common thing in both is their joint probability. So, their discrepancy depends on $P(x)$ and $P(y)$. If these two probabilities are equal or close to each other then according to formulas, the conditional probabilities are close to each other. This also depends on the degree of dependence between $x$ and $y$.

For example consider the events of $T:\text{Thunder}$ and $B:\text{Black clouds}$. They often come together and they strongly depend on each others, so approximately you may write:

$$ P(T) \approx P(B) \approx P(T \cap B)$$

Then the conditional probabilities are equal:

$$P(B | T) \approx P(T | B) \approx 1 $$

Now consider the example of $H:\text{Being hanged}$ and $D:\text{Death}$, which one is the cause of the other one but not vice versa. We can safely assume whenever one is hanged, they die, and it's the only usage of hanging. So:

$$ P(H) \approx P(D \cap H)$$ Therefore: $$P(D | H) = \frac{P(H \cap D)}{P(H)} \approx 1$$

However, death could be due to many other reasons than being hanged and $$P(D) \gg P(D \cap H)$$

Therefore: $$P(H | D) = \frac{P(H \cap D)}{P(D)} \approx 0$$

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.