It should be said, I have no background in calculus, but have read a few books on infinity and still don't understand it's relation to statistics.

From what I've read it sounds like the textbook answer is something along the lines of:

"The probability of picking 7 is indistinguishable from 0"

Which seems to read as though it's impossible to pick 7. And of course 7 can be swapped out with any number in the infinite set. Which leads me to the conclusion that the probability of picking any number from an infinite set is indistinguishable from 0. This, though, seems to be able to be flipped on it's head by simply asking what the probability of picking a number contained by the set via randomly picking some positive integer.

Can someone help me make sense of this?

  • 2
    $\begingroup$ Because this question appears to involve intuition rather than any rigorous concepts or definitions, it seems pertinent to ask this: what exactly do you mean by "picking" a number? What process does that involve? Is it intended to be a physical one? If not, then what is your mathematical description of it? $\endgroup$ – whuber Dec 28 '15 at 15:56
  • $\begingroup$ Random picking. $\endgroup$ – click here Dec 28 '15 at 16:11
  • 2
    $\begingroup$ The probability of picking 7 from a hat of 10 numbers is 1/10. The probability of picking 7 from a hat of 1,000 numbers is 1/1,000. The probability of picking 7 from a hat of 1,000,000 numbers is 1/1,000,00. As the denominator approaches infinity (aleph-0), the probability approaches 0. Such that, as far as I understand: The probability of picking 7 from a hat of aleph{0} is 1/aleph{0} which I understand to mean that to be a probability indistinguishable from 0. We can ofcourse run the gamut by switching any number with 7. $\endgroup$ – click here Dec 28 '15 at 16:51
  • 4
    $\begingroup$ Taking a limit or referring to infinite quantities is no longer a physical process; it's necessarily a mathematical one. In this light, please note that there is no axiom of probability asserting such a limit of a sequence of probability distributions must be a probability distribution, while there are axioms that directly imply there can be no uniform probability distribution on a countable set. This makes it difficult to discern what your question actually is--whether it's about physical processes, limits, axioms of probability, your intuition, or something else. $\endgroup$ – whuber Dec 28 '15 at 17:54
  • 2
    $\begingroup$ "Probability 0" and "impossible" are different things. Also, there is no uniform distribution (in any standard sense of the term) on a denumerable set, but there are plenty of non-uniform distributions. $\endgroup$ – Meni Rosenfeld Dec 28 '15 at 19:37

It's totally a question of what probability distribution you're putting on this set (which I'm guessing is either the integers or natural numbers in your example). It can't be uniform because $\aleph_0$ refers to countable infinities and a sum of countably-many of the same nonnegative number is either zero or infinite (it can never equal one).

But all you need to do is spread out the probability in another way. Some simple examples are the standard discrete distributions you encounter like the Poisson or geometric (as whuber has already pointed out). If $X$ is our selection from $\mathbb{N}$ say, then in the Poisson case we take $P(X = k) = e^{- \lambda} \lambda^k / k!$ for some $\lambda > 0$, and in the geometric case (depending on how we decide to define things) we have $P(X = k) = (1 - p)^k p$ where $p \in (0, 1)$. In both cases $k$ can be any natural number whatsoever, so we have no trouble selecting 7 or any other value. The problem only comes in when we insist on trying to choose things uniformly at random.

  • $\begingroup$ The problem is that the set has inifinite measure, not that it has infinite cardinality. The interval [0, 1] is uncountable, but we can select from it uniformly using the usual metric because it has finite measure. $\endgroup$ – Dietrich Epp Dec 28 '15 at 21:10
  • $\begingroup$ @DietrichEpp I didn't say the problem came from the set having infinite cardinality, but from being countably infinite. $\endgroup$ – dsaxton Dec 28 '15 at 21:24
  • $\begingroup$ That doesn't change the fact that a countably infinite set can still have nonzero finite measure. $\endgroup$ – Dietrich Epp Dec 28 '15 at 21:46
  • $\begingroup$ @DietrichEpp Yes, like one of the examples in my post. What is your point? $\endgroup$ – dsaxton Dec 28 '15 at 21:53
  • 1
    $\begingroup$ @clickhere Because for any discrete distribution (one with finite or countable support) the total probability must add to one to be a valid probability distribution. What happens if you take $p + p + p + \ldots$ and so on forever? Can this sum ever be one? $\endgroup$ – dsaxton Dec 29 '15 at 19:38

When you start playing with infinities phrases like "indistinguishable from 0" are not the same as "equal to 0." In this case the probability is being handled as a limit. One could consider the probability of 7 being picked is $1/\aleph_0$, however, $\aleph_0$ is what is called a "limit cardinal," and infinitesimals generated from limit cardinals are not included in the set of real numbers, so we have to be creative.

Consider a series defined by $1/x$ as we vary x:

x    1/x
1    1
2    0.5
3    0.3333...
4    0.25
10   0.1
100  0.01
1000 0.001
ℵ_0  ?

So what's $1/\aleph_0$? It can't be zero, because that doesn't satisfy the rules of arithmetic $0 * \aleph_0 = 0$, but basic reorganization of the algebra shows that $1/x * x = 1$ for all x ($x \neq 0$).

The concept of a limit is used to rigorously explore this intuitive understanding that there is some limiting thing out there for $1/x$ that equals 0. In very intuitive and imprecise wording, this limit is the number "approached" by a series, but typically never actually achieves it.

Thus, when given an infinite number line and asked to pick a random value from it, the probability of any given number being picked is very very small. In fact, so small that the real numbers can't even describe how small it is. To capture this concept of smallness, we often look at the limit of the probability of a number being picked as we grow the set towards a cardinality of $\aleph_0$. This limit is zero. This is not to say the probability of picking any given number is zero, just that any process used to describe the meaning of picking randomly from an infinite set that starts from smaller sets and builds larger ones from there (typically using induction) will describe a series of probabilities that get closer and closer to 0 as the sets get larger.

The more formal definition of this limit could be phrased with some of the calculus you mention, using the epsilon-delta definition of a limit. By this definition, if you pick any arbitrarily small "epsilon," and try to determine which sized sets result in a probability of picking 7 below this epsilon, I can identify some set size for which all larger sets must yield probabilities smaller than this epsilon bound.

Of course, you probably don't need the formal definition much. Sometimes intuition is enough, but its good to know there's a formal definition coming later down the line for you!


Think about it this way: Randomly picking a number means that you will have a number (some number) after picking. If it were true that it's impossible to pick the number N from the infinite set, then any number you would have picked would lead to a logical contradiction.

So the real question is whether you can choose integers at random with an even distribution from an infinite set. I'd say the answer is no, not in 'reality'. In order to do so you would need a process by which you have an equal probability of selecting any number from an infinite set. This would require (at the very least) an infinite amount of time and unfortunately, that is not available to us.

For example, let's say you roll a 10 sided die (0-9) some number of times to pick a random number. How many times you roll it will set the upper bound of what number you might get. If you roll it three times, your number will come from the set 0-999. You'll be rolling the die forever since there is no limit to the number of digits in the set.

  • $\begingroup$ This argument is faulty, because the potentially infinite process of rolling that die (1) will almost surely terminate and (2) can even have a finite expected number of rolls, depending on what you are attempting to simulate with it. This answer overall may mislead many readers by confusing 'random' with 'uniform'. $\endgroup$ – whuber Dec 29 '15 at 2:03
  • $\begingroup$ @whuber Because it must terminate, you cannot pick from an infinite set. That's the point. If you look at the second comment by the original poster, he is describing uniform probability. A non-uniform distribution where the probability of selecting 'larger' numbers drops to zero isn't really relevant to what was asked i.e. if you flip a coin until heads comes up and the number of flips determines the number selected, the odds of selecting seven is definitely distinguishable from 0. $\endgroup$ – JimmyJames Dec 29 '15 at 15:13
  • $\begingroup$ You misread my statement: it is not the case that the die-rolling procedure must terminate. It can proceed forever. It simply is a fact that the probability of termination is $1$. $\endgroup$ – whuber Dec 29 '15 at 15:15
  • $\begingroup$ What would cause it to terminate? The process I described must not terminate in order to work. You seem to be confirming my point but seem to think you are contradicting it. $\endgroup$ – JimmyJames Dec 29 '15 at 15:21
  • $\begingroup$ @whuber Let me go at it from a different direction. If we assume the process must terminate for some reason (e.g. heat death of the universe) then we are saying that there must be only N rolls. No matter what N is, there will then be an infinite set of numbers that are greater than or equal to larger than 10^N that have a probability of being selected of exactly zero. The numbers smaller than 10^N have a non-zero probability of selection. So in order for the distribution to be uniform, it must not terminate. If the probability of termination is 1, then the process is inherently flawed. $\endgroup$ – JimmyJames Dec 29 '15 at 15:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.