Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
In order to get more fundamental in my understanding of probability I watched mathematicalmonk's lectures involving $\sigma$-algebras etc. - good. One of my main concerns was to better understand the basis for conditional probability: $P[A|B] = P[AB]/P[B]$. My question is simple
How do we know that this quotient is indeed a probability measure?
After all, things like the ``Odds'' $P[A] / P[\bar{A}]$ is not a probability measure.
Assume that $P(B) > 0$ and define $P(A\mid B) = \frac{P(A\cap B)}{P(B)}$ for all events $A$ in the $\sigma$-algebra $\mathcal F$ over which $P(\cdot)$ is a probability measure. Note that we are assuming that $B \in \mathcal F$ (else $P(B)$ would not be defined). Then $P(\cdot\mid B)$ also is a probability measure on the same $\sigma$-algebra $\mathcal F$.
Axiom I: $P(A\mid B) \geq 0$ for all events $A \in \mathcal F$.
This should be obvious since $P(A\cap B) \geq 0$. Those who take Axiom I as $0 \leq P(A) \leq 1$ should note that since $(A\cap B) \subset B$, we have that $P(A\cap B) \leq P(B)$ and so $P(A\cap B) \leq 1$ as needed.
Axiom II: $P(\Omega\mid B) = 1$
This should also be obvious since $(\Omega \cap B) = B$ and so $P(\Omega\mid B) = 1$.
Axiom III: For any countable sequence of disjoint events $A_1, A_2, \ldots$, in $\mathcal F$ $$P\left(\left. \bigcup_{n=1}^\infty A_n \,\right\vert \,B\right) = \sum_{n=1}^\infty P(A_n \mid B)$$
This requires just a little bit of work. Begin by noting that $A_1\cap B, A_2\cap B, \ldots$ is a countable sequence of disjoint events. Then,
$$\begin{align*}P\left(\left. \bigcup_{n=1}^\infty A_n \,\right\vert \,B\right) &= \frac{P\left(\left(\bigcup_{n=1}^\infty A_n\right)\cap B\right)}{P(B)}\\ &= \frac{P\left(\bigcup_{n=1}^\infty (A_n\cap B)\right)}{P(B)}\\ &= \frac{\sum_{n=1}^\infty P(A_n\cap B)}{P(B)}\\ &= \sum_{n=1}^\infty P(A_n \mid B) \end{align*}$$
The key motivation for the notion of probability is the long-term relative frequency phenomenon (LTRF). Please forget mathematics when reading the following sentence: let ${\cal E}$ be an experiment which is repeated infinitely many times and independently, in that no issue influences others, and let $A$ be an event related to the possible issues of ${\cal E}$; then the relative frequency of the occurences of $A$ converges to some number. The LTRF is considered as a "law of nature", not a mathematical statement; in particular, the notion of "independence" is not a mathematical one here; it intuitively means that no issue influences others. And as I said the LTRF is the key motivation for probability: $\Pr(A)$ is the mathematical modelling of the limit number mentioned in the LTRF.
The LTRF clearly motivates the basic additivity axiom of probability: $\Pr(A \cup B) = \Pr(A)+\Pr(B)$ for disjoint events $A$ and $B$. Indeed, considering the LTRF context and denoting by $N(A,n)$ the number of occurences of an event $A$ in the first $n$ independent performances of ${\cal E}$, one clearly has $N(A\cup B, n) = N(A,n)+N(B,n)$ and $\frac{N(A\cup B, n)}{n} \to \Pr(A\cup B)$ whereas $\frac{N(A, n)}{n} \to \Pr(A)$ and $\frac{N(B, n)}{n} \to \Pr(B)$. In fact the "basic" additivity holds from $N(\cdot, n)$. By "basic" I mean "finite". The theoretical $\sigma$-additivity is a purely mathematic notion.
The LTRF also motivates the notion of conditional probability $\Pr(B \mid A)$. A "by-product" of the conditional probability is the notion of independence (i.e. the mathematical modelling of the intuitive notion of independence mentioned in the LTRF): $A$ and $B$ will be said to be independent when $\Pr(B\mid A)=\Pr(B)$. Now the conditional probability is introduced as follows in the LTRF context: the conditional probability $\Pr(B \mid A)$ is the long-term proportion of experiments for which $B$ occurs among those experiments for which $A$ occurs. That is, $\Pr(B \mid A)$ is considered as the "LTRF limit" of $\frac{N(A\cap B, n)}{N(A,n)}$. Note that $\frac{N(A\cap B, n)}{N(A,n)}= \frac{N(A\cap B, n)/n}{N(A,n)/n}$, and by the LTRF rule it yields the mathematical definition $\Pr(B \mid A)=\frac{\Pr(A\cap B)}{\Pr(A)}$.
This fundation shows that the conditional probability $\Pr(\cdot \mid A)$ must be a probability. I know that does not answer your mathematical question, but I hope that helps you to familiarize yourself with the notion of conditional probability.
The answer to your question is not difficult: it is easy to see that $B \mapsto \frac{\Pr(A\cap B)}{\Pr(A)}$ satisfies the axioms of probability. In fact the $\sigma$-additivity works for $B \mapsto \Pr(A\cap B)$, and the other axioms obviously hold true.
D. Williams. Weighing the Odds: A Course in Probability and Statistics. Cambridge University Press, 2001.
I am not a mathematician, but a probability measure has to sum to $1$ when added over all events of $A$ given $B$:
$$\displaystyle \sum_{A} \mathbb{P}(A \lvert B) = 1$$
and that does not hold for the odds, for example.
In other words, the axioms of probability must hold.
