4
$\begingroup$

I know that if events $A$ and $B$ are independent, then $P(A\cap B) = P(A).P(B)$.

Lets say I roll a die, and define the following events:

$x$ = Number is a multiple of $3 = \lbrace 3, 6\rbrace$

$y$ = Number is more than $3 = \lbrace 4, 5, 6\rbrace$

$z$ = Number is even = $\lbrace 2, 4, 6 \rbrace$

Therefore,

$P(x) * P(y)$: 0.166666666667

$P(x \cap y)$ : 0.166666666667

$P(x) * P(z)$ : 0.166666666667

$P(x \cap z)$ : 0.166666666667

$P(y) * P(z)$ : 0.25

$P(y \cap z)$ : 0.333333333333

My question is, why are $xy$ and $xz$ independent events but $yz$ is not? I had expected all combinations of events to be independent as neither events affect any other.

$\endgroup$

2 Answers 2

5
$\begingroup$

To add to the Rob's answer the joint probability of two events in the discrete case can be decomposed as follows (See the wiki's definition for the discrete case) :

$P(A \cap B) = P(A|B) P(B)$

Therefore, if the fact that we observe the event $B$ is not informative about the probability of our observing the event $A$ then it must be the case that:

$P(A|B) = P(A)$.

Thus, if the two variables are independent (i.e., the occurrence of one event does not tell us anything about the other event) then it must be the case that:

$P(A \cap B) = P(A) P(B)$

The crucial condition for independence of two events is: $P(A|B) = P(A)$. This condition is essentially stating that the "Probability of our observing event A over the unrestricted sample space = Probability of our observing the event A if the sample space is restricted to the set of possible outcomes compatible with the event B". Thus, the knowledge that the event B occurred has no information about the probability of observing the event A.

$\endgroup$
5
  • $\begingroup$ Thanks Rob and Srikant for your explanation. I understand independent events now. Just out of curiosity, if I were to draw the Venn Diagram of aforementioned events, are there any characteristics that I can look for to reliably identify independent events? $\endgroup$
    – Sara
    Oct 22, 2010 at 8:15
  • $\begingroup$ @Sara No, You cannot. Venn diagrams will only help you write down the formulas for the various possible relationships between the events (e.g., $P(A \cup B) = P(A) + P(B) - P(A \cap B)$. But, the diagram itself will not tell you anything about the probabilities of the events under consideration. $\endgroup$
    – user28
    Oct 22, 2010 at 9:34
  • $\begingroup$ @Srikant: Noted. Thank you so much for your assistance. $\endgroup$
    – Sara
    Oct 22, 2010 at 9:44
  • 1
    $\begingroup$ @Sara, if you look at a probability density function en.wikipedia.org/wiki/Probability_density_function of the two events fxy then you will get a rectangular distribution if they are independent. $\endgroup$
    – Kortuk
    Oct 22, 2010 at 15:18
  • 1
    $\begingroup$ @Sara, the wiki is not perfect for the multivariable case. There are better sources. but a prob density function of 2 variable can be very interesting. $\endgroup$
    – Kortuk
    Oct 22, 2010 at 16:15
4
$\begingroup$

I find it easiest to think of independence in terms of conditional probabilities: $X$ and $Y$ are independent if $P(Y|X)=P(Y)$ and $P(X|Y)=P(X)$. That is knowing one does not change the probability of the other. In this example,

\begin{aligned} P(Y|X) &= P(Y \cap X) / P(X) = 1/2 = P(Y)\\ P(Y|Z) &= P(Y \cap Z) / P(Z) = 2/3 \ne P(Y)\\ P(Z|X) &= P(Z \cap X) / P(X) = 1/2 = P(Z) \end{aligned}

So knowing $X$ does not affect the probability of either $Y$ or $Z$. But knowing $Z$ does affect the probability of $Y$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.