$P[X=x]=0$ when $X$ is a continuous variable

Question

I know that for any continuous variable $P[X=x]=0$.

But I can't visualize that if $P[X=x]=0$, there is an infinite number of possible $x$'s. And also why do their probabilities get infinitely small ?

Answer 1

Probabilities are models for the relative frequencies of observations. If an event $A$ is observed to have occurred $N_A$ times on $N$ trials, then its relative frequency is $$\text{relative frequency of }(A) = \frac{N_A}{N}$$ and it is generally believed that the numerical value of the above ratio is a close approximation to $P(A)$ when $N$ is "large" where what is meant by "large" is best left to the imagination (and credulity) of the reader.

Now, it has been observed that if our model of $X$ is that of a continuous random variable, then the samples of $X$ $\{x_1, x_2, \ldots, x_N\}$ are $N$ distinct numbers. Thus, the relative frequency of a specific number $x$ (or, more pedantically, the event $\{X = x\}$) is either $\frac 1N$ if one of the $x_i$ has value $x$, or $\frac 0N$ if all the $x_i$ are different from $x$. If a more skeptical reader collects an additional $N$ samples, the relative frequency of the event $\{X=x\}$ is either $\frac{1}{2N}$ or continues to enjoy the value $\frac 0N$. Thus, one is tempted to guess that $P\{X = x\}$ should be assigned the value $0$ since that is a good approximation to the observed relative frequency.

Note: the above explanation is (usually) satisfactory to engineers and others interested in the application of probability and statistics (i.e. those who believe that the axioms of probability were chosen so as to make the theory a good model of reality), but totally unsatisfactory to many others. It is also possible to approach your question from a purely mathematical or statistical perspective and prove that $P\{X = x\}$ must have value $0$ whenever $X$ is a continuous random variable via logical deductions from the axioms of probability, and without any reference to relative frequency or physical observations etc.

Answer 2

Let $(\Omega,\mathscr{F},P)$ be the underlying probability space. We say that a measurable function $X:\Omega\to\mathbb{R}$ is an absolutely continuous random variable if the probability measure $\mu_X$ over $(\mathbb{R},\mathscr{B})$ defined by $\mu_X(B)=P\{X\in B\}$, known as the distribution of $X$, is dominated by Lebesgue measure $\lambda$, in the sense that for every Borel set $B$, if $\lambda(B)=0$, then $\mu_X(B)=0$. In this case, the Radon-Nikodym theorem tells us that there is a measurable $f_X:\mathbb{R}\to\mathbb{R}$, defined up to almost everywhere equivalence, such that $\mu_X(B)=\int_B f(x)\,d\lambda(x)$. Let $B=\{x_1,x_2,\dots\}$ be a countable subset of $\mathbb{R}$. Since $\lambda$ is countably additive, $\lambda(B)=\lambda\left(\cup_{i\geq 1}\{x_i\}\right)=\sum_{i\geq 1}\lambda(\{x_i\})$. But $$ \lambda(\{x_i\}) = \lambda\left(\cap_{k\geq 1}[x_i,x_i+1/k)\right) \leq \lambda\left([x_i,x_i+1/n)\right) = \frac{1}{n} \, ,\qquad (*) $$ for every $n\geq 1$. Due to the Archimedean property of the real numbers, since $\lambda(\{x_i\})\geq 0$, the inequality $(*)$ holds for every $n\geq 1$ if and only if $\lambda(\{x_i\})=0$, entailing that $\lambda(B)=0$. From the assumed absolute continuity of $X$ it follows that $\mu_X(B)=P\{X\in B\}=0$.

Answer 3

$X$ is a continuous random variable means its distribution function $F$ is continuous. This is the only condition we have but from which we can derive that $P(X = x) = 0$.

In fact, by continuity of $F$, we have $F(x) = F(x-)$ for every $x \in \mathbb{R}^1$, therefore: $$P(X = x) = P(X \leq x) - P(X < x) = F(x) - F(x-) = 0.$$

Answer 4

This is really a question about probabilities.

It could be rephrased something like

Suppose there are an infinite number of disjoint events. Why must their probabilities eventually become arbitrarily small?

Well, if they didn't become small, there would be a positive number $\epsilon$ smaller than all those probabilities. One axiom of probability implies the probability of a union of a finite number of distinct events is the sum of their probabilities. Since "infinite" means there exist finite subsets of any size $n,$ we at least know that for any whole number $n$ there is a set of $n$ distinct events of probability $\epsilon$ or greater. Taking $n \gt 1/\epsilon$ (guaranteed by the Archimedean property of real numbers) we would deduce the existence of a collection of disjoint events whose probability exceeds $n\epsilon\gt 1,$ an obvious impossibility. This completes the proof: we are obliged to conclude the original assumption is false: namely, the probabilities do become arbitrarily small.

BTW, it should be no mystery why an infinite number of disjoint events can all have zero probability. Take the simplest kind of random variable: the constant $X=0.$ For any nonzero $x,$ obviously $\Pr(X=x) = 0.$ There are a lot of such $x$! The problem really is,

how can it be possible that $\Pr(X=x)=0$ for all possible values $x$ without forcing all probabilities to be zero?

The formal mathematical answer is buried in the probability axioms: they do not state, nor even imply, that the probability of an event must be the sum of the probabilities of all its elements. This is the subtlety in the restriction to countable unions and countable sums. The real numbers are not countable.

But then how do you visualize such a thing? The cumulative distribution function $F_X$ of any random variable $X$ is a useful tool. By definition, for any number $x,$ $F_X(x)$ is the chance that $X \le x,$ written $\Pr(X\le x).$ We can (and usually do) draw a graph of $F_X.$ The probability axioms imply its heights must lie between $0$ and $1$ (they are all probabilities, after all) and that the graph never decreases as you increase $x.$ (This latter is proven by noting that for $y\gt x,$ $F(y)-F(x) = \Pr(y \lt X \le x) \ge 0.$)

Using the CDF, then, we find that the probability of $X=x$ is the amount by which the graph of $F$ rises at $x$ as you move left to right. In any continuous such graph, all such rises are zero (that's essentially the definition of "continuous"). Nevertheless, despite rising by an amount $0$ at every point $x,$ somehow the graph manages to increase its elevation from $0$ to $1$ throughout its domain. See Wikipedia (at "CDF") for some examples. Pondering this visually evident fact might help you appreciate these issues better. For a more insight, study Zeno's Paradoxes.

Answer 5

There are many answers here which address the technical reasons for why infinitesimally small slices of a continuous probability distribution, $Pr(X=x)$, are 0. The key idea here is that this probability is uncountable.

Instead of adding to the mathematically driven answers here, I would like to connect this phenomenon to a real-world example (with a frequentist approach):

Imagine you are analyzing the (gaussian) distribution of temperatures for a particular time at a particular place. Let's say noon, Jan 1st, in Cambridge. Population: $\mu = 0$ and $\sigma = 10$

You can comfortably say that 95% of sample temperatures means will fall between ~ $-20$ and $20$. Shrink down that range and recompute your conf. interval. Rinse and repeat. Eventually, you are left with the interval of $-0.000001$ and $0.000001$.

What percentage of sample means would fall into that interval? A very small percentage in a technical sense, but in reality, most people would assign it $0$% upon some thought. How would you even measure such a precise temperature? As you continue to shrink the interval until both the positive and negative limits approach $0$, you'd need to find more and more precise measurement tools. Eventually, you'll run out of precision.

In truth, it's almost never going to be exactly 0 degrees at noon, and even if it were, you wouldn't have any way to measure it.

Thus, we assign zero probability to infinitesimally specific events in a continuous distribution.

Answer 6

This question is very simple. PDF is the density, then to get the probability you need to multiply it by the width of the region. So, when you get a smaller region around the point of interest $x$ the height of the density doesn't change, hence the probability is smaller and smaller while you squeeze the area around your point until it becomes exactly zero.

The difficult question is the reverse one, like this: https://stats.stackexchange.com/a/273407/36041