4
$\begingroup$

I am reading the Probability Lifesaver and in the introduction to continuous variables it shows the contradiction when summing infinite number of zeros such as the union of signleton events as,

enter image description here enter image description here

I can understand the concept that we cannot meaningfully sum uncountable events, or that the union of uncountable events is not necessarily the sum of their probabilities. But thereafter,

enter image description here

Why? Why can it be any number from 0 to 1?

$\endgroup$
1
  • 4
    $\begingroup$ The statement at the end -- "an uncountable sum of zero can be any finite number from 0 to 1" -- is mathematical nonsense. There's no logically consistent way to define uncountable sums, that's all. Take this quotation as an effort to supply some intuition, perhaps. (There is a sense in which "uncountable" sums can be defined -- in the cases where all but a countable number of the terms are zeros! But in that sense an uncountable sum of zeros is always zero by definition and clearly that's not what's meant here.) $\endgroup$
    – whuber
    Nov 19, 2020 at 18:07

3 Answers 3

3
$\begingroup$

The problem here is that there isn't really any concept of an "uncountable sum" for you to have an intuition about! Summation is initially defined as a binary operation, then extended to finite sums by induction, and then extended to countable sums by taking limits. That is as far as it goes. The closest analogy we have to a "sum" over an uncountable set is the integral. We can make some valid probability statements involving the integral that are similar to the quoted claim, and involve integration as a kind of "uncountable sum". Below I will show what you can and can't validly say.


What we can say

Using integration as our version of the "uncountable sum", there is a rough analogy that holds here in probability theory that mimics the intuitive properties put forward in the quote section. Suppose we have a continuous random variable $X$ with quantile function $Q_X$ and density function $f_X$. Then the norming property of probability gives:

$$\int \limits_\mathbb{R} f_X(x) \ dx = 1.$$

For any $x \in \mathbb{R}$ we have $\mathbb{P}(X=x) = \int_x^x f_X(x) \ dx = 0$ and yet we can get any quantile value $0 \leqslant p \leqslant 1$ by taking:

$$\int \limits_{-\infty}^{Q_X(p)} f_X(x) \ dx = p.$$

This means that the "uncountable sum" (actually an integral) over a bunch of outcomes with zero probability can give us any probability value we want between zero and one. This occurs because our "sum" is really an integral that is "summing" an infinite number of infinitesimally small values. That is essentially what this kind of "intuitive" description is pointing to.


What we can't say

Unfortunately, it is not possible to extend this idea to get an exact analogy to the quoted section. Even taking the integral as our concept for an "uncountable sum", an exact analogy to the quoted section would be something like the following invalid result:

$$\ \ \int \limits_{-\infty}^{Q_X(p)} \mathbb{P}(X=x) \ dx = p. \quad \quad \quad \text{(Invalid equation)}$$

The reason this does not work is that $\mathbb{P}(X=x) = \int_x^x f_X(r) \ dr \neq f_X(x)$. When we extend from the countable domain to the uncountable domain, and start dealing with "uncountable sums" as integrals, we start using infinitesimals. These infinitesimals are small enough that they give zero probability when we integrate over a single value, but they are actually larger than zero, so when we integrate over a larger (uncountable) set of them, they can give a positive value.

Consequently, we can see that the result really comes down to the fact that, once we start using an "uncountable sum" we also need to start using infinitesimals, which are infinitesimally small non-zero values. If we were to translate the quote into a strictly valid observation on mathematics, it would say that even though infinitesimal values look like zero from one perspective (e.g., integrating them over a single value), an integral of infinitesimal values can be any non-zero value.

$\endgroup$
1
$\begingroup$

An example for which $\text{Prob}(X\in[a,b])$ should be able to take any value between 0 and 1 is a uniform distributed variable. In that case if $X \sim \mathcal{U}(0,1)$ then for $0\leq a\leq b\leq1$ you have $$\text{Prob}(X\in[a,b]) =b-a$$

This is a contradiction with

$$\text{Prob}(X\in[a,b]) =\sum_{\lbrace x \in \mathbb{Q} | 0 \leq x \leq 1 \rbrace} 0 = 0$$


A similar question on maths.stackexchange is the following:

Why is $\infty \cdot 0$ not clearly equal to $0$?

The answer there argues in a similar way, if you would treat multiplication of zero and infinite as if you would treat multiplication of finite numbers, then you could argue not just that $\infty \cdot 0 = 0$ but just as well that $\infty \cdot 0 = \infty$, $\infty \cdot 0 = 1$ or anything else.

The solution to the paradox is in considering whether the terms of the operation are correctly used. What is $0$ and what is $\infty$? Are they correctly used, or are they misinterpreted due to some ambiguities?


Steven Miller writes

"Our argument crucially used that the probability of a sum of disjoint events is a sum of the sum of probabilities of the events. This is true if it is a finite sum, or even a countably infinite sum, but not necessarily true if it's an uncountable sum."

So the problem isn't actually in the infinite sum. The issue with the infinite sum of singleton events is not in the uncountable property. Note: we could have a countable infinite sum of all events $X=x$ when we consider rational numbers $\mathbb{Q}$ (which are countable) as the singleton events.

The argument about the infinite sum is not wrong.

Instead, the error is in the definition of a probability space composed of only singleton events with each a zero probability. The problem is that by introducing an event space of singleton events, each with zero probability, we are creating a measure space that does not have total measure 1.

There is nothing wrong with defining a measure space where each rational number $x\in \mathbb{Q}$ between $0$ and $1$ has some constant measure $m\geq 0$. It is just not a probability measure. This is because for $m=0$ the total measure is zero, and for $m>0$ the total measure is infinite.

$\endgroup$
7
  • $\begingroup$ That probability is not an uncountable sum, though: the axioms don't give us any right to suppose that. $\endgroup$
    – whuber
    Nov 19, 2020 at 18:10
  • $\begingroup$ It is not: indeed, the whole point of the quoted passage is that such sums make no sense. One has to find a different approach -- and that way lies measure theory. $\endgroup$
    – whuber
    Nov 19, 2020 at 18:17
  • $\begingroup$ It still doesn't hang together, because basic (defining) properties of summation would imply, say, that the sum of the zero probabilities in the intervals $[0,1]$ and $[1,2]$ would be equal and therefore the sum in the interval $[0,2]$ would be $2.$ There's nothing to limit the value of such a sum. In a comment to the question I maintained that this "heuristic" is just a bad one -- thus, it would be better to avoid it than to try to explain it, because you will inevitably be embroiled in contradictions. $\endgroup$
    – whuber
    Nov 19, 2020 at 19:24
  • $\begingroup$ your answers and comments are fantastic. Thank you very much! $\endgroup$
    – ECII
    Nov 19, 2020 at 21:03
  • 1
    $\begingroup$ @whuber yes, I meant a constant measure $m$ for every single $\lbrace x \in \mathbb{Q} | 0 \leq x \leq 1 \rbrace$. $\endgroup$ Nov 20, 2020 at 14:36
1
$\begingroup$

It's important to realize that probability, by its definition/construction as a mathematical concept, is only countably additive and not "uncountably additive." Generally in mathematics there is no such concept as uncountably additive (there are a few contexts where uncountable sums have been defined, but they aren't applicable to interpreting probability as an uncountable sum, and in those contexts an "uncountable sum of zeros" is always zero). Really, addition always only involves a finite number of terms and any infinite sum is defined in more complicated ways (e.g. as a limit of finite sums).

It could be problematic that in the shared text the author says first "we cannot assert the probability of an uncountable union of disjoint events is the sum of their probabilities" (which is true) and then says "an uncountable sum of zeros can be any number between 0 and 1." The last statement is a reasonable informal/intuitive way to think about it but an "uncountable sum of zeros" isn't generally a valid mathematical object and definitely isn't valid in this context. Maybe later in the text that is clarified. I would worry that the average reader would see this and just be perplexed, thinking that uncountable sums of zeros are a coherent mathematical thing in other contexts and can be equal to whatever we want (in a mathematically coherent way). The concept of an "uncountable sum of zeros" is simply not defined generally speaking (it is defined in some contexts, but always gives a sum of zero then).

It's important to understand that any time we are talking about "uncountable sums of zeros" in this context of probability, we are talking informally and intuitively and not about formally existing mathematical objects. Generally, it is important to distinguish between informal ideas and formal mathematical objects with the latter being described according to established axioms, definitions, etc.

In the context of probability (and measure in general), if we are to think of adding up uncountably many zeros to get the probability of an event, then that would result in uncountable sums of zeros giving any number between 0 and 1. So that is an informal intuitive reason why uncountable sums of zeros don't make sense here. There is no formal mathematical object, an "uncountable sum," that we are referring to here (i.e. it's just not defined in this context).

Consider the Cantor set $C$ and the set of irrationals $A=[0,1]\cap\mathbb Q^c$. If we are considering $X$ to be a uniformly distributed random variable in $[0,1]$, then $P(X\in C)=0$ and $P(X\in A)=1$ (the Lebesgue measure of these sets is firmly established, and that is identical to their probability under the uniform distribution). So these probabilities are not well-defined as uncountable sums of zeros. Similarly we can create an uncountable set of any probability between 0 and 1 that will similarly be thought of as giving an uncountable sum of zeros equal to its probability. E.g. the fat Cantor set $C_{1/4}$ with middle bits of length $1/4^n$ removed at each stage has measure $1/2$ gives $P(X\in C_{1/4})=1/2$. We can construct fat Cantor sets of any measure between 0 and 1, and each is uncountable.

$\endgroup$
4
  • $\begingroup$ Mathematicians generally disagree with you that an "uncountable sum of zeros" cannot be defined. See math.stackexchange.com/questions/20661/…, for instance. $\endgroup$
    – whuber
    May 2, 2021 at 16:26
  • 1
    $\begingroup$ @whuber I didn't say it "cannot" be defined in some contexts, but just that it isn't defined "generally" (in most contexts). And when "uncountable sums of zeros" are defined, they equal zero. We could take this to be the "general mathematical definition" of the concept, sure. The uncountable sums the textbook in the OP is speaking of indeed are not defined though. I assume that it is never actually defined in this textbook at least, if it is, then that would presumably answer the question posed. $\endgroup$
    – jdods
    May 2, 2021 at 17:09
  • $\begingroup$ @whuber I've tried to clarify this a bit. If you still see problems, please let me know. $\endgroup$
    – jdods
    May 2, 2021 at 17:30
  • $\begingroup$ Thank you for that (+1). $\endgroup$
    – whuber
    May 2, 2021 at 18:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.