7
$\begingroup$

I and my friends just had a little discussion whether the events are independent or dependent if they have no outcome in common. I thought that they have to be independent. When two events are independent, then $P(A)=P(A\mid B)$. Is the information given in the question enough to establish this?

If you look at it as a Venn diagram, then if there is no overlap between A and B, then they are independent. But my friend objected and said that this depends on the sample space.

So let's say we have two events: $P(A)=6/12$, $P(A\mid B)=2/4$, and $P(B)=4/12$, then obviously $P(A\mid B)$ equals $P(A)$. But for me, something smells fishy here.

I know this is more a stochastic question that a statistic question, but maybe someone can help.

$\endgroup$
5
  • $\begingroup$ What do you mean by "no result in common"? $\endgroup$
    – Peter Flom
    Commented Mar 19, 2012 at 11:21
  • $\begingroup$ Adam, I made minor edits to your post. Please, check that I didn't alter the meaning of the 3rd paragraph. $\endgroup$
    – chl
    Commented Mar 19, 2012 at 12:07
  • 7
    $\begingroup$ From your description, it seems like you might be confusing the idea between independent and mutually exclusive. $\endgroup$
    – King
    Commented Mar 19, 2012 at 12:38
  • 1
    $\begingroup$ if i got it right, no result in common=mutually exclusive. But the question was formulated a little sloppy. So maybe that is what makes the answering of this question a little complicated.It boils down to the fact that mutually exclusive does not say anything about the probabilites occuring. So it can be both.THANKS FOR THE GREAT ANSWERS BTW! $\endgroup$
    – Adam
    Commented Mar 19, 2012 at 17:57
  • $\begingroup$ @Adam: You got it right. $\endgroup$
    – Neil G
    Commented Mar 20, 2012 at 6:47

6 Answers 6

12
$\begingroup$

No, events with no result in common are not independent if the events come from the same sample space.

An example: Throw a single fair die. Let event A be 'throw is a 1', and event B be 'throw is a 2'. Then $P(A) = P(B) = 1/6$, but $P(A|B) = P(B|A) = 0$, as the die throw can't be both 1 and 2.

$\endgroup$
3
  • 5
    $\begingroup$ You are correct for nonsingular countable spaces, but it would be good to take a little care with the generality of your first statement. After all, many sample spaces are infinite and many nonempty events have zero probability. They provide counterexamples: such events may have no outcome in common, yet they are independent. $\endgroup$
    – whuber
    Commented Mar 19, 2012 at 17:22
  • 1
    $\begingroup$ @whuber: maybe you should add a precise answer? $\endgroup$
    – Neil G
    Commented Mar 20, 2012 at 1:14
  • 1
    $\begingroup$ @whuber - You could increase the generality of the statement by noting that dependence between any two propositions can only occur when both are marginally possible and at least one is not marginally certain. That is $A$ and $B$ can only be dependent if both $p(A)>0$ and $p(B)>0$ and $p(A)+p(B)<2$. This is just the logical result of "a false proposition implies all propositions, true and false" and "a true proposition implies all true propositions". $\endgroup$ Commented Mar 20, 2012 at 11:58
10
$\begingroup$

Independence can be thought of as "If event A occurs it tells you nothing about the probability of event B occurring".

However, if two events share nothing in common then clearly if I know that one event occurred then I know that the other event COULDN'T have occurred. So I am getting information about the other event so there the two events can't be independent.

Note that this was an intuitive explanation and it's possible for B to be empty in which case A and B can be mutually exclusive and still be independent as explained in some of the other answers.

$\endgroup$
5
  • $\begingroup$ That's true as long as neither of the events contains no outcomes. Such an event would be both disjoint and independent of all other events. $\endgroup$
    – Neil G
    Commented Mar 20, 2012 at 1:13
  • $\begingroup$ That's very true and a good point. I was going for a more intuitive reason why mutually exclusive events aren't independent but you're right and I should note that in my answer. $\endgroup$
    – Dason
    Commented Mar 20, 2012 at 3:58
  • $\begingroup$ Okay, the first paragraph is good, but the second paragraph isn't right. $\endgroup$
    – Neil G
    Commented Mar 20, 2012 at 6:15
  • $\begingroup$ @NeilG - the second paragraph is correct, because if $A$ and $B$ are mutually exclusive, then we have the logical truths $A\implies\overline{B}$ and $B\implies\overline{A}$. In probabilistic notation this means $p(\overline{A}|B)=p(\overline{B}|A)=1$. This means that as long as $A$ is not impossible when $B$ is false and/or $B$ is not impossible when when $A$ is false we always have dependence between mutually exclusive events. $\endgroup$ Commented Mar 20, 2012 at 11:24
  • 1
    $\begingroup$ @probabilityislogic: No :) See my answer. $\endgroup$
    – Neil G
    Commented Mar 20, 2012 at 11:49
5
$\begingroup$

It is true that if $P(A)=P(A\mid B)$ then the two events are intependent.

Now we know that: $P(A\mid B)=\frac{P(A\cap B)}{P(B)}$. Note the intersection in the numerator. If the two events didn't have "anything in common" then we would have $P(A\cap B)=0$ since the set $A\cap B$ would be empty. However independence works in the space of probabilities. Two events are called independent if $P(A\cap B)=P(A)P(B)$, ie the probability of having both events occuring is equal to the probability of the first one occuring times the probability of the second one. The fact that event $A$ occured doesn't tell us anything about event $B$.

If on the other hand $P(A\cap B)=0$ this simply says that the two events cannot happen at the same time: they are disjoint.

Example 1. What is the probability of throwing a die and getting both 1 and 3 at the same time? Since the two events are disjoint, we have that this probability is zero.

Example 2. What is the probability of throwing a die two times and getting 1 in the first time and 2 in the second? The two events are not disjoint; the fact that one happened doesn't exclude the other. However they are independent: the fact that one happened tells us nothing about the other. The probability of getting 1 in the first throw is 1/6, the probability of getting 2 in the second throw is 1/6. Then (abusing notation a bit): $P(2\mid 1)= \frac{P(1 \cap 2)}{P(1)} = \frac{ \frac{1}{36}}{\frac{1}{6}}=\frac{1}{6}=P(2)$.

$\endgroup$
3
$\begingroup$

You are confusing dependence with mutual-exclusivity. Two events $A$ and $B$ are independent if and only if $P(A \cap B) = P(A)P(B)$. They are disjoint if and only if $A \cap B = \emptyset$.

Consider a normally-distributed random variable $X$ with mean 0 and variance 1. Here is a pair of events for each of the four categories:

  1. the events $0 < X < 1$ and $1 < X < 2$ are dependent and disjoint
  2. the events $X = 0$ and $1 < X < 2$ are independent and disjoint
  3. the events $0 < X < 1$ and $0 < X < 2$ are dependent and not disjoint, and
  4. the events $X < 0$ and $\lvert X \rvert < 3$ are independent and not disjoint.

If events $A$ and $B$ are independent and disjoint then \begin{align} 0 &= P(A \cap B) \\\\ &= P(A)P(B) \\\\ &\implies P(A) = 0 \vee P(B) = 0. \end{align}

$\endgroup$
16
  • 1
    $\begingroup$ @probabilityislogic: You don't need any limits. The standard definition of an event space always includes at least one event with no outcomes because sigma algebras must be closed under intersection. I have added an event for case 4 where neither event is certain. The definition of independence is $P(A)P(B) = P(A \cap B)$. $\endgroup$
    – Neil G
    Commented Mar 20, 2012 at 12:09
  • 1
    $\begingroup$ @probabilityislogic: But the event X=0 is not "false." And, it's not the result of a limit. It's a proper event in the Borel algebra on the reals. $\endgroup$
    – Neil G
    Commented Mar 20, 2012 at 14:19
  • 1
    $\begingroup$ @probabilityislogic: Of course you can observe it. Just because its probability is zero, it doesn't mean that it's impossible. en.wikipedia.org/wiki/Almost_surely $\endgroup$
    – Neil G
    Commented Mar 20, 2012 at 15:44
  • 1
    $\begingroup$ @probabilityislogic: What you're doing is rederiving the concept of a sigma algebra. What you'll discover is that if you have events $X \le b$ and $X > b$, then you must have events like $X=b$ since the space of events must be closed under countable intersection since we want to be able to ask the probability of a conjunction of events. Above you wrote that "But note that so long as e>0 we will still have dependence"; the problem with this argument is that properties that hold for all terms in an infinite sequence do not necessarily hold at the limit of the sequence. $\endgroup$
    – Neil G
    Commented Mar 21, 2012 at 5:07
  • 1
    $\begingroup$ @NeilG Yes, your post is greatly improved and I am up-voting it. One minor comment, though. $A$ and $B$ being disjoint implies that $P(A \cap B) = 0$, but the converse is not true, and so your "if and only if" assertion (third sentence of first paragraph) is not correct. I have posted a separate answer too. $\endgroup$ Commented Mar 21, 2012 at 13:45
2
$\begingroup$

Let $A$ and $B$ denote two events defined on a sample space $\Omega$. The formal definition of independent events is as follows.

Definition: $A$ and $B$ are said to be (stochastically) mutually independent events if

$$P(A\cap B) = P(A)P(B).$$

It is easily shown that any one of the four relations shown below implies the other three:

$$\begin{align*} P(A\cap B) &= P(A)P(B)\\ P(A^c\cap B) &= P(A^c)P(B)\\ P(A\cap B^c) &= P(A)P(B^c)\\ P(A^c\cap B^c) &= P(A^c)P(B^c) \end{align*}$$ and so if $A$ and $B$ are mutually independent events, then so are $A^c$ and $B$ mutually independent events, as are $A$ and $B^c$, and $A^c$ and $B^c$.

Now, if $P(B) > 0$ so that we can write $P(A \mid B)$ as $P(A\cap B)/P(B)$, then $P(A\mid B)$ equals $P(A)$, and this is often taken as the colloquial meaning (or definition) of independence. $A$ and $B$ are independent events if knowing that $B$ has occurred does not change our estimate of the probability of $A$. Put another way, the posterior probability $P(A\mid B)$ is the same as the prior probability $P(A)$. The asymmetry in the colloquial definition even leads people to say $A$ is independent of $B$ (which can make beginners wonder whether $B$ is independent of $A$ or not), but the formal definition makes it clear that independence is a mutual property: one cannot have $A$ independent of $B$ but $B$ dependent on $A$.


Turning to the OP's question, if $0 < P(A), P(B) < 1$, then mutual independence and mutual exclusion are mutually exclusive properties. If one property holds, the other cannot. Of course, the most commonly encountered case is that neither property holds. Said out loud and clear

  • If $A$ and $B$ are mutually independent events, then they cannot be mutually exclusive events.

  • If $A$ and $B$ are mutually exclusive events, then they cannot be mutually independent events.

In the first case, note that mutual independence implies that $P(A\cap B) = P(A)P(B) > 0$ and so the intersection of $A$ and $B$ has positive probability. In the second case, $P(A\cap B) = 0$ cannot equal $P(A)P(B)$ since neither $P(A)$ nor $P(B)$ is $0$ by assumption and so their product is a positive number.

As a corollary, note that $A$ and $A$ cannot be a pair of mutually independent events. and nor can $A$ and $A^c$ be mutually independent events.


Much of the discussion in the comments has centered on the rare cases when $P(A)$ or $P(B)$ happen to equal $0$ or $1$. First note that since $$P(A \cap \Omega) = P(A) = P(A)P(\Omega)$$ and so $A$ and the certain event $\Omega$ are independent events for all choices of $A$. Similarly, since $$P(A \cap \emptyset) = P(\emptyset) = 0 = P(A)P(\emptyset),$$ $A$ and the impossible event $\emptyset$ are independent events for all choices of $A$. More generally, if $B$ is an event of probability $0$ (not necessarily the impossible event), then since $A\cap B$ is a subset of $B$ and hence also has probability $0$, we can generalize to $P(A \cap B) = 0 = P(A)P(B)$ and so

  • Any event of probability $0$ is independent of all events (including itself and its complement). If $B$ is an event of probability $0$, then $B$ and $B^c$ are independent events that are mutually exclusive.

If $B$ is an event of probability $0$, then $B^c$ is an event of probability $1$. Since $B$ and $A$ are independent events for all choices of $A$, so also are $B^c$ and $A$ independent events for all choices of $A$. Thus, we have

  • Any event of probability $1$ is independent of all events (including itself and its complement). If $A$ is an event of probability $1$, then $A$ and $A^c$ are independent events that are mutually exclusive.

Note that, as @NeilG has pointed out in his answer, if $A$ and $B$ are independent events that are mutually exclusive, then at least one of $A$ and $B$ must be an event of probability $0$.

We also have an anticorollary: $A$ and $A$ are mutually independent events if and only if $P(A)$ equals either $0$ or $1$. $A$ and $A^c$ are mutually independent events if and only if one of $P(A)$ and $P(A^c)$ equals $0$ (and the other equals $1$.)

$\endgroup$
-1
$\begingroup$

mutually exclusive events.. natural blonde hair, and black skin. If I know someone has black skin, I know they will not have naturally blonde hair. Therefore, the characteristic of hair color and skin color are dependent.. Knowing that someone has black skin tells me information about what their hair will be like.. Their hair will NOT be blonde.. These things are dependent. Mutually exclusive implies dependence!

independent events... IQ and shoe size.. If you thought someone was a size 8 shoe, would it change your opinion about their IQ if I told you their shoe size was actually a 10? No.. Shoe size and IQ (likely) have no relationship. (Assume it's true.) Shoe size and IQ are independent of one another.

I hope that helps.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.