# Probability of continuous random variable

It is claimed that probability of continuous R.V. is 0 for ALL x in R. That is, for every point the probability is zero.

But, somehow when we sum all these zero probabilities over the entire domain $\mathbb{R}$, the total becomes equal to 1.

Can somebody explain how 0s can add up to 1? This defies kindergarden math.

In fact, what you sum up to become 1 is not the probabilities of the variable being each of the values: you sum up (more or less) the probabilities of the variable being within an infinitely small interval around each value. While the limit of these individual probabilities is also zero, the limit of their sum need not be (this is only so because there is an infinite number of summands, so it is not easy to create an example of this).

There is a difference between an integral and a sum.

• I see what you mean. Then, i think the proper definition should be that probability of any particular point is an extremely small number ( 1 / |R| ), so that the sum of probabilities of all points in R is equal to 1. – Saideira Oct 14 '11 at 15:23
• @Saideira What value would you like to use for this extremely small number $1/\vert R \vert$? Equivalently, according to you, how many points are there in $R$? – Dilip Sarwate Oct 14 '11 at 15:31
• Not really. If you divide a cake of size one (in whatever unit of cake size) under an infinite number of children, they are all going to be extremely unhappy, because they will receive exactly zero units of cake. With regards to infinity, the cake is a lie. FYI: there is a branch of mathematics that does treat this infinitely small number as nonzero (infinitesimal analysis). I strongly advise you to accept 'regular' maths and calculus first, though. – Nick Sabbe Oct 14 '11 at 15:33
• "If you divide a cake of size one (in whatever unit of cake size) under an infinite number of children, they are all going to be extremely unhappy, because they will receive exactly zero units of cake." Well, you are glossing over the issue of countable versus uncountable, but it is perfectly possible to slice the cake in half, give one piece to the first child, slice the remainder in half and give one piece to the second child, and continue in this fashion. Maybe you should say that it is impossible to divide the cake into an infinite number of portions of equal size greater than $0$. – Dilip Sarwate Oct 14 '11 at 16:48
• It is pedagogically unsound to divide a cake in unequal parts: all children are created equal and thus they should get equal parts of the cake. Just kidding. You're right, of course (though countability has nothing to do with it: if you have a noncountable number of children, you can still give half the cake to the first one). – Nick Sabbe Oct 14 '11 at 17:00

Kindergarden math does not incorporate the concept of infinity, let alone distinguishes the different orders of magnitude of infinity. (Kindergarden math does not even include negative numbers, and if you are stuck with the kindergarten math, you may never understand why you owe money on a house, let alone understand how compound interest works.)

A density of a random variable is a Radon-Nikodym derivative of the measure implied by the random variable with respect to Lesbegue measure. It does take an infinity for this thing to work out, as the finite summation of zeroes will still be zero, as you correctly noted. There are far more complicated things involved here, including measurability that relies on "good" sets of a real line being closed under countable unions, intersections and complements.

Without real analysis, these things are difficult to grasp. Are there more numbers in rational numbers than in natural numbers? (No, the cardinalities of the two sets are the same.) Are there more numbers on the real line than in natural numbers? (Yes, they are different types of infinities.) Are there more numbers on a unit square than there are on the real line? (No, the cardinalities are the same.) The proofs for each of these statements can be easily constructed... but again the kindergarten math does not incorporate the concept of a proof, just taking the teacher's word for it ;)

• So where can I find the proof that infinite number of 0s add up to something other than 0? I'd love to see it! – Saideira Oct 14 '11 at 14:41
• @Saideira By definition, the sum of a sequence of numbers is computed by tracking the sequence of its partial sums: if that sequence converges to a value, that value is the sum. (Otherwise the sum does not exist.) By induction, the sum of a finite number of zeros is always zero. Thus, the sequence of partial sums of an infinite sequence of zeros is $(0,0,\ldots,0,\ldots)$ which (obviously) converges to $0$, QED. – whuber Oct 14 '11 at 15:58
• Thanks, whuber. The issue here is that the total probability is an integral rather than a sum, either finite or countable. I guess the discussion about it went on with Nick Sabbe's answer, as well. – StasK Oct 14 '11 at 18:51

Suppose I am in the business of building mathematical models for clients and a client gives me the outputs from his "random number generator" that has generated a million random numbers $x_1$, $x_2, \ldots, x_{10^6}$ all between $0$ and $1$, and all different. He would like to have a model for the output of this random number generator as a random variable $X$.

First attempt: $X$ is a discrete random variable taking on values $x_1$, $x_2, \ldots, x_{10^6}$ with equal probability. That is indeed very satisfactory. It matches the data perfectly, the client is happy, and so am I as I pocket my fee.

But the next day, the client is back because he has run his random number generator some more and none of the new outputs $x_{10^6+1}$, $x_{10^6+2}, \ldots$, match what the model is predicting, viz., the output will be one of $x_1$, $x_2, \ldots, x_{10^6}$ (with equal probability). So, now I have to think a bit.

Second attempt: I look at the histogram of the numbers generated by the random number generator and it looks pretty flat all the way across $(0,1)$. It looks like $X$ can take on any real number value between $0$ and $1$ and so I model $X$ as a continuous random variable. But what value should I assign to, say, $P\{X = 0.2173333605\}$? It appears just once in the list and thus has relative frequency $10^{-6}$ on the first million outcomes but looking at $x_{10^6+1}$, $x_{10^6+2}, \ldots$, I see that the relative frequency is decaying away towards $0$. I look at $2\times 10^6$ outputs and the relative frequency of the value $0.2173333605$ has halved, while the relative frequency of $0.92387634504$ is stuck at $0$ because it is not among the first two million trials.

So since probabilities in the model should be similar to observed relative frequencies in real life, I say $P\{X = x\} = 0$ for every real number $x$, $0 < x < 1$. Wait a minute! Where did the probability disappear? Well, the probability is hiding in the intervals. Roughly $500,000$ of $x_1, \ldots, x_{10^6}$ are in the interval $(0, 0.5)$ while roughly $1,000,000$ of $x_1, \ldots, x_{2\times 10^6}$ are in the interval $(0, 0.5)$. In other words, the relative frequencies of the intervals are pretty stable but the relative frequency of a specific number is either $0$ or decaying away towards $0$. Similar remarks apply to other intervals. So the model for this continuous random variable $X$ is that

• For any $a, b$ such that $0 \leq a < b \leq 1$, $P\{a < X < b\} = b-a$.
• For any $a, b$ such that $a \leq 0 < b \leq 1$, $P\{a < X < b\} = b$.
• For any $a, b$ such that $0 \leq a < 1 \leq b$, $P\{a < X < b\} = 1-a$.
• For any $a, b$ such that $a \leq 0 < 1 \leq b$, $P\{a < X < b\} = 1$.

More succinctly, for $a < b$, $P\{a < X < b\} = \min\{b, 1\} - \max\{a, 0\}$, and $X$ is said to be uniformly distributed on $(0, 1)$. Its probability density function is $$f_X(x) = \begin{cases} 1, & 0 < x < 1,\\0, & \text{otherwise}.\end{cases}$$ Note that the function is nonnegative and the area under the curve is $1$. Also $$P\{a < X < b\} = \int_a^b f_X(x) dx.$$ More generally, a continuous random variable $Y$ has a probability density function $f_Y(y)$ that is nonnegative and has area $1$, and $$P\{a < Y < b\} = \int_a^b f_Y(y) dy.$$

As @NickSabbe and StasK have told you, for continuous random variables, probabilities are obtained via integrals not via sums. Indeed, it is possible that $f_Y(y) > 1$ for some values of $y$. As long as $$P\{-\infty < Y < \infty\} = \int_{-\infty}^{\infty} f_Y(y) dy = 1$$ it is perfectly OK to have $f_Y(y) > 1$ for some values of $y$. The density of the probability can exceed $1$, but the total probability is $1$. Note that $f_Y(y) > 1$ will definitely break the sums that you want to use.