# Intuition behind $P(\{\omega\})=0$ in uncountable sample spaces with continuous distributions

It would be nice to carry on thinking of probability as a simple "aggregate" of the long-term frequency associated with each possible elementary outcome, $P(\{\omega\})$, so that if a fair die has a probability of $1/6$ associated with each face, the probability of the event "even outcome", or the subset of ${\omega}$ consisting of $\{2,4,6\}$ is just $3*P(\{\omega\})=1/2$. In this way the probability space $(\Omega,\mathscr F,\mathbb P)$ is clearly defined when $\mathscr F= 2^\Omega$.

The need to steer away from this comfortable approach seems to arise from the fact that when transitioning from a countable sample space to an uncountable one, such as the $[0,1]$ interval, we can't do the same with each possible outcome - i.e. it is not possible to define the probability space of the uniform distribution for any subset of $[0,1]$, i.e. the power set $2^{[0,1]}$, because the collection of sets $2^{[0,1]}$ is simply too large (?). In fact the probability of any singleton is $P(\{\omega\})=0$ for continuous distributions (leaving aside discrete/continuous mixed distributions expressed sometimes making use of the Dirac delta function).*

Excepting possible misstatements in the preceding background framing of the question, it seems fair to say that measure theory gets its cue at this point. Yet the explanations as to why the naive approach to probability as frequency of individual outcomes is inadequate seems to be hidden in the last chapter of the book. So would it be possible to get a "spoiler alert", preferably in plain English of the problems behind the questions:

Why is the probability of a singleton zero over an uncountable sample space? Why does the naive concept of probability breaks down if we try to define it as the distance between two points ($b-a$) in the case of the uniform distribution in any possible subinterval within $[0,1]$? And, possibly, How do Borel sets are a solution that makes sense to circumvent this problem?

From "Probability Essentials" by Jacod and Protter to clarify the background of the question:

[*] NOTE The last part of this sentence ("continuous distributions") should have been there from the beginning, but it wasn't - it's the circular problem of "showing" that steps have been taken to look for an answer when asking a question. So, to be clear, Matthew's answer is completely congruent with the OP, in which I came short of asking the question properly (and probably I still am).

• The problem is not related to the size of the power set--that has nothing to do with anything. Ultimately, measure theory was forced on mathematicians at the end of the nineteenth century when they wanted to perform countable summing operations, as in the theory of trigonometric series, and needed an integral that behaved "nicely" with respect to such sums and related limiting processes. – whuber Jan 5 '16 at 4:41
• On page 35 of the Second Edition of "Probability Essentials" by Jacod and Protter, I came across the sentence, "The collection of sets of $2^{[0,1]}$ is simply too large for this to work." – Antoni Parellada Jan 5 '16 at 4:50
• One reason that argument has no bearing on this situation is that the problem occurs even for a countable space, such as the set of rational numbers in $[0,1]$: it is still not possible to define a uniform distribution. I suspect the meaning of "this" in your quotation may refer to something slightly different than what you describe here. In fact, the language in this post seems to confuse a probability space with a distribution, which could be another issue you might want to address. – whuber Jan 5 '16 at 4:56

The probability of a singleton is not necessarily zero in an uncountable measure space. You can, and often may, assign non zero probabilities to singleton sets, they are usually called point masses.

For example, a simple modification of lebesgue measure $\mu$ on $[0, 1]$ like

$$\nu(S) = .5 + .5\mu(S) \ \text{if} \ 0 \in S$$ $$\nu(S) = .5\mu(S) \ \text{if} \ 0 \notin S$$

is easily checked to still be a measure. This measure has $\nu(\{0\}) = .5$.

On the other hand, it is true that not too many singletons can have non-zero probability. Particularly, only a countable number of point masses can exist. Here is the standard proof of this fact.

The need to steer away from this comfortable approach seems to arise from the fact that when transitioning from a countable sample space to an uncountable one, such as the [0,1] interval, we can't do the same with each possible outcome - i.e. it is not possible to define the probability space of the uniform distribution for any subset of [0,1]...

This is a bit muddled, but upon second reading I understand what your are trying to express. The precise statement is something like this:

There is no function $\mu: 2^{[0, 1]} \rightarrow [0, 1]$ which we would call a measure in the following sense:

1) $\mu([0, 1]) = 1$

2) For any disjoint collection of sets $A_{\alpha}$ (indexed by $\alpha$), the measure of the union of the collection is equal to the sum of the measures.

3) If $T$ is a translation, then the measure of $T(A)$ (modulo 1) is the same as the measure of $A$.

The proof of this is actually rather simple, here's a sketch. Start with the rational numbers $Q$. For any irrational number $a$, the set

$$Q + a = \{q + a \mid q \in Q \}$$

is a translation of $Q$, so $\mu(Q \cap [0, 1]) = \mu((Q + a) \cap [0, 1])$ by point 3 (and some hand waving about wrapping around). Use the axiom of choice to pick irrational numbers, one after another, until you have a collection $a_{\alpha}$ so that

$$[0, 1] = \cup_{\alpha} ((Q + a_{\alpha}) \cap [0, 1])$$

If the measure of $Q \cap [0, 1]$ is $0$, then applying 2 shows that the measure of $[0, 1]$ must also be $0$, which can't happen by 1. On the other hand, if the measure of $Q \cap [0, 1]$ is some positive quantity, then applying 2 again shows that the measure of $[0, 1]$ must be infinite. In both cases we reach a contradiction, so no such measure can exist.

This is why Lebesgue chose countable additivity for his measures, dropping that requirement results in the failure of the above argument (do you see it?), and one can construct self consistent measures using this scheme (but it is hard to do so).

Why does the naive concept of probability breaks down if we try to define it as the distance between two points (b−a) in the case of the uniform distribution in any possible segment within [0,1]?

It doesn't really (except for, as @whuber points out, the conflation of measures and distributions). This is essentially the way you define Lebesgue measure, or at least the start of doing so. The issue with stopping here is you cannot, for example, say what the measure of $Q$ is. The idea of borel sets is to expand your collection of intervals to a larger collection of sets that are measurable in a self consistent way.

• I don't know if it's the same concept, but I have seen the delta function used for point probabilities in some examples motivating the topic. – Antoni Parellada Jan 5 '16 at 4:29
• Yes, that's correct. Depending on how serious you want to be, this is either a convenient notation, or a really deep object of mathematical analysis. – Matthew Drury Jan 5 '16 at 5:52

Maybe a more 'intuitive' explanation is as follows: if we define $P([a,b]) = b-a$ then, by definition $P(\{a\})=a-a=0$.

Obviously the interval [0,1] is the (uncountable) union of all the points in it or $[0,1]=\cup_{x\in [0,1]} \{x\}$.

Therefore we would like the $P(\{x\})=0$ to add up to $P([0,1])=1$, but adding up zeros to get 1 does not work with a countable number of terms ...