How do we characterize probabilities on this infinite sample space

Question

Consider the following set, $S$, of infinite sequences:

$$S = (a_n)_{n \in \mathbb{N}} \\\text{where } a_n \in \mathbb{N} \text{ and } a_n \leq nk+1 \text{ with }k\text{ a positive integer}$$

in which all sequences have equal measure.

These sequences are equivalent to rolling a dice infinitely often, where the size of the dice is increasing each step according to $kn+1$ in the n-th step.

---

My interest is in the fraction of sequences with infinitely many 1-s.

I have two conflicting ideas about this, and therefore I am not so sure about my formalism and whether I made errors:

T1: set theory tells us that the fraction of sequences with infinitely many 1-s must be zero, since for every infinite sequence $(a_n)$ with infinitely many 1-s we can construct infinitely many other sets of sequences by shifting the sequence values one up like $a_{n+1}=a_n+1$, resulting in a finite number of 1-s

on the other hand

T2: If we calculate the expected value of 1s in the sequences of $S$ then we get to $\sum_{x=1}^{\infty}{\frac{1}{x}}$ (in the simple case of $k=1$) which is infinite. Does this mean that we can say that most sequences have infinitely many 1s?

---

Now the question here is about gaining more understanding of such infinite sampling space. A wrong application of this sample space results in paradoxical results and erroneous statements. How do we handle this correctly, and what are (correct) intuitive ways to view this space?

1. While I understand this question becomes rather vague at the moment. My aim is to gain more insight into the infinite size of the sampling space and what it's effect is on theorems and axioms that are more particular suitable for finite sampling spaces.

2. To make the question more specific: Let's say we want to find out the probability that a random point in $S$, is within the subset $S_{RL} \in S$, which contains all the sequences with infinitely many 1s.

   What is the probability and how do we do get to this?

Answer 1

[Note: I think this answer is correct. If it's wrong, then hopefully at least it will be a starting point for further discussions.]

It's not possible to talk about the "fraction of sequences with infinitely many 1s" unless you have some way of defining "fraction". This can't be done consistently without using measure theory.

In measure theory terms, there is no way of defining a measure on this space such that every subset is measurable. In everyday terms, if you can talk about "the fraction of sequences with property $P$" for every choice of $P$, then you will reach a contradiction.

Measure theory issues cannot be avoided by saying "all sequences have equal measure" as this would restrict you to using either the zero measure or the counting measure. In either case, you wouldn't have a probability space because there is no way that the measure of the entire space could be equal to $1$.

Fortunately, there is only one way to define a sensible measure on this space, and under this measure, the probability that a sequence contains infinitely many 1s is $1$. This is reasonable, because certainly everyone would agree that the probability of getting infinitely many heads when flipping a coin infinitely many times is $1$, and this is a very similar situation.

How is the measure defined? To define a measure, you need to specify a sigma-algebra of measurable subsets and a measure on them. Fortunately, in this case it is possible to take a short cut since we are exactly in the situation of Theorem 7.16 in this pdf. (A finite space is a metric space under the discrete metric in which the Borel sets are all the subsets of the space.) [Edit: I have just noticed that the theorem in that pdf is for infinite copies of a single space rather than different spaces. But I think it is still true, for example, see the Kolmogorov Extension Theorem article on Wikipedia, under 'General Form'.]

The resulting product measure is defined as follows: For a finite list $b=(b_1, b_2, \ldots, b_n)$ of numbers, define

$$S_b = \{ s \in S: s_i = b_i, 1\le i \le n\}.$$

Then define the measure of $S_b$ by

$$\mu(S_b) = \prod_{i=1}^n P(a_i = b_i) = \prod_{i=1}^n \frac{1}{ik+1}$$

and the theorem tells us that this magically extends to a probability measure on all measurable subsets of $S$.

Now we have to use the fact that $\mu$ is a measure to get $\mu(S_{RL})=1$. The complement of $S_{RL}$ is

$$\cup_{n=1}^\infty \cap_{k=n}^\infty A_k$$

where $A_k$ is the subset of $S$ consisting of those sequences for which $a_k \neq 1$. This shows that $S_{RL}$ is a measurable set because its complement can be expressed in terms of $A_k$, which are measurable because they can be expressed in terms of $S_b$'s using countable unions, intersections and complements. Now,

$$\mu(\cap_{k=n}^\infty A_k)=0$$

because $\mu(\cap_{k=n}^N A_k) \rightarrow 0$ as $N \rightarrow \infty$. (Edit: coveniently, I think this is actually the calculation given in the other answer.) Therefore, the measure of the complement of $S_{RL}$ is $$\mu(S_{RL}^C)= \mu(\cup_{n=1}^\infty \cap_{k=n}^\infty A_k) = 0$$ because the measure of a countable union of sets of zero measure is zero, by properties of measure, and so $\mu(S_{RL})=1$ because $\mu$ is a probability measure by construction.

Answer 2

I would say that to ask this question, we must ask the probability that any sequence has finite ones. I think it's sufficient to show that the probability of generating a sequence, as you described, that has finite ones, by rolling a dice at each step, is zero. If that's true then drawing any element of $S$ should guarantee to give you an element in $S_{R L}.$ $$ \prod_{j=1}^\infty \left( 1 - \frac{1}{k j + 1} \right) = \prod_{j=1}^\infty \frac{k j}{k j +1} = 0. $$

It's not obvious that the above product is zero, but it is shown (and I'm borrowing this result from) Chapter 2 Example 6a of this book, which was brought up to ask (and answer) the question in this recent thread.

Now it's easy to show that the above product, with a finite upper bound is nonzero, simply by the fact that none of the factors are individually zero. So it follows that, $$ \prod_{j=m}^\infty \frac{k j} {k j + 1} = 0 $$ for any positive integer $m.$

Now let's calculate the probability of a sequence in $S$ having exactly $n$ zeros. To do this, we define $W_n$ to be the set of all subsets of $\mathbb{N}$ of size $n$. (For example, elements of $W_3$ can be $\{1,4,9 \}, \{2, 888, 1.5 \times 10^{55} \},$ etc.) Then the probability of having exactly $n$ ones is,

$$ P(\text{n ones}) = \sum_{s \in W_n} \left( \prod_{j \in s} \frac{1}{k j + 1} \prod_{j \notin s} \frac{k j }{k j + 1} \right). $$

We can decompose the second product in the summand to look like this, $$ \prod_{j \notin s} \frac{k j}{k j + 1} = \prod_{j \notin s; j < \max(s)} \frac{k j}{k j +1} \prod_{j = \max(s)+1}^\infty \frac{k j} {k j + 1}, $$ where $\max(s)$ is the highest integer in the set $s.$ Remember, we're summing over all subsets of $\mathbb{N}$ of size $n,$ so $s$ is a sequence of $n$ positive integers. It follows that $\max(s)$ is finite for all sets $s$ in the sum. Thus $P(\text{n ones})$ is a sum of terms, each of which are zero.

Therefore $P(\text{n ones}) = 0.$ This is true for all positive integers $n.$ It follows that all sequences must have infinite ones.