10
$\begingroup$

I know this is duplicated but I think the question is a bit different and needs different answer.

How can CDF be continuous and have derivative at each point that is not equal to zero but the probability at each point is zero?

Why not say for example if you want to choose a real number between 0 and 1 that the probablity is 1/N? Where this is just infinitesimal. (Just an example to describe the situation.)

It can neither be finite nor zero

The difference is: Why not assign it an infinitesimal instead of zero and to say that an event with P = 0 can happen?

$\endgroup$
13
  • $\begingroup$ What does N stand for? I do not follow your logic and I don't see how you have really made this a different question. $\endgroup$ Apr 12, 2017 at 17:55
  • 2
    $\begingroup$ There is a continuum of real numbers between 0 and 1. It is not only infinite it is uncountably infinite. $\endgroup$ Apr 12, 2017 at 18:02
  • 3
    $\begingroup$ This is really a question of understanding the mathematics (calculus) that statistics & probability are built on, rather than primarily a probability question. It might help you to read this Mathematics SE question: Is dy/dx not a ratio? $\endgroup$ Apr 12, 2017 at 18:11
  • 2
    $\begingroup$ A computer program cannot draw any real number in $[0,1]$. It can't even draw any rational number, only the set of possible rational numbers that can be represented in the finite number of bits assigned to the variable. Can you name me any real world example in which we draw a number from a continuous distribution, whose value is known exactly? $\endgroup$ Apr 12, 2017 at 19:04
  • 1
    $\begingroup$ Please read the first third of Edward Nelson's pretty little monograph, Radically Elementary Probability Theory. $\endgroup$
    – whuber
    Apr 12, 2017 at 22:01

3 Answers 3

14
$\begingroup$

A guess at your point of confusion:

Zero probability does not mean an event cannot occur! It means the probability measure gives the event (a set of outcomes) a measure zero.

As @Aksakai's answer points out, the union of an infinite number of zero width points can form a positive width line segment and similarly, the union of an infinite number of zero probability events can form a positive probability event.

More explanation:

  • Our intuition from discrete probability is that if an outcome has zero probability, then the outcome is impossible. If the probability of drawing the ace of spades from a deck is equal to zero, it means that ace of spades is not in the deck!

  • With continuous random variables (or more generally, an infinite number of possible outcomes) that intuition is flawed.

    • Probability measure zero events can happen. Measure one events need not happen. If an event has probability measure 1, you say that it occurs almost surely. Notice the critical word almost! It doesn't happen surely.
    • If you want to say an event is impossible, you may say it is "outside the support." What's inside and outside the support is a big distinction.
    • Loosely, an infinite sum of measure zero events can add up to something positive. You need an infinite sum though. Each point on a line segment has zero width, but collectively, they have positive width.
$\endgroup$
9
  • 2
    $\begingroup$ "Zero probability does not mean impossible" - I always had trouble with this. I'd say if it happened then probability is not zero. $\endgroup$
    – Aksakal
    Apr 12, 2017 at 20:15
  • 1
    $\begingroup$ @Aksakal: Suppose that you flip a fair coin repeatedly, stopping when it lands heads. I make two claims: (a) it is possible that you could keep on flipping the coin forever, with the coin landing tails infinitely many times; (b) the probability that the coin lands tails infinitely many times is zero. With which one do you disagree? $\endgroup$
    – wchargin
    Apr 12, 2017 at 20:21
  • 1
    $\begingroup$ I would disagree with b, I think people only say zero because it is the most suitable number in real numbers, it should be a number that satisfies e> 0 where e should be really close to zero. The only number that may satisfies that is zero. That is why I argue that infinitesimal are better $\endgroup$ Apr 12, 2017 at 20:38
  • 2
    $\begingroup$ @wchargin You can't observe infinite number of heads $\endgroup$
    – Aksakal
    Apr 12, 2017 at 20:40
  • 1
    $\begingroup$ @user106860 Saying a probability measure assigns an event a measure of zero is equivalent to saying the event almost surely will not occur. Zero probability (along with law of large numbers and some more argument) indeed implies that your $\lim_{N \rightarrow \infty} \frac{\#5}{N} = 0$, but that doesn't explain what's going on. Aksakal's answer gets more at the heart of the matter. The concepts in play here are really concepts from set theory and real analysis. $\endgroup$ May 16, 2019 at 12:49
12
$\begingroup$

It's really not a statistics question. It's a real analysis question. For instance, it's almost the same as asking "what's the width of a point in line?" (the answer is zero, by the way)

This is an interesting situation though. In mathematics the line is defined as a set of points. There are certain geometric constraints on the points, so that they form a line and not a circle, for instance. However, that's not what's important.

What's important is this. If the width of each point is zero, and the line is a set of points, how come the sum of widths of all its points is NOT zero? You add two zeros and it gives you a zero. If I keep adding this way shouldn't the length of a line be zero? Apparently, not!

This is the same question you're asking. How is it that each point's probability is zero, yet the total probability is one? The reason why is this question the same is because probabilities are intimately linked to the concept of the length of a line between two points. The central concept of the modern probability theory is the concept of a measure. Unsurprisingly, it has its roots in the simplest of all measures: the length in geometry.

If you want a shortcut in understanding this bind boggling stuff then look up the concept of countable and uncountable sets. Note the difference between infinite countable sets and uncountable sets. Both have infinite number of points in them, yet the latter has more points in it (totally crazy!). So discrete and continuous random variables (and their distributions) are related to these two kinds of sets.

UPDATE
Example: In English there are countable and uncountable nouns such as apple vs. milk. I could ask you how much does an apple weigh? And you could say that it's half a pound in this batch. However, if I asked how much does milk weigh, it wouldn't make a sense without specifying an amount such as a pint or a quart.

In this regard the discrete random variables and their probabilities are like apples and their weights. You could say that the probability of Poisson variable 1 is 10%, for instance.

The continuous random variables are like milk. It's pointless asking what's the probability of a given value, you need to specify the bucket. Say, for a standard normal (Gaussian) variables you could ask what's the probability that their values are between 0 and 1, and the answer would be something like 34%. However, the probability of 1 is pretty much meaningless in practical sense. You can calculate the density at $x=1$ but what are you going to do with it? It's not the probability. In the same way if you're interested in the weight of milk the density of milk is not an answer, you need to specify the container size then we can tell you the weight using its density. That's why probability density function is actually called density, it originates from densities of bodies.

$\endgroup$
9
  • $\begingroup$ Really really interesting argument..This is the dark side of mathematics. "almost the same" but in a sense points should have zero width and probabilities shouldn't. I don't really know how you can define a length as the sum of width of points, I don't think that it is a good definition...? $\endgroup$ Apr 12, 2017 at 20:21
  • 2
    $\begingroup$ @user3733086, the width of a point on a curve and the probability density have the same math behind. $\endgroup$
    – Aksakal
    Apr 12, 2017 at 20:39
  • $\begingroup$ And there is no solution to this paradox?? $\endgroup$ Apr 12, 2017 at 20:53
  • 1
    $\begingroup$ @user3733086, it's not a paradox. It's one of the first things they teach in real analysis, then continue in measure theory. I'm not sure how to explain it intuitively. I was just trying to point out that it's not a stat question really hoping that you'd look at the math behind it $\endgroup$
    – Aksakal
    Apr 12, 2017 at 21:01
  • $\begingroup$ @user3733086 The solution is to stop thinking measure zero is synonymous with impossible and start thinking that measure zero simply means something has measure zero. For example, the Lebesgue measure of the open interval $(2, 3)$ is 1. The Lebesgue measure of the closed interval $[2,3]$ is also 1 (even though the latter is a superset of the former). The Lebesgue measure of the point $2$ and the point $3$ is zero. $\endgroup$ Apr 12, 2017 at 21:02
0
$\begingroup$

I think it is helpful to imagine the area under the point. The probability for a continuous distribution is the integral of the PDF from (a,b). If you pick a single point (a,a) is there any area? Imagine a simple PDF like the uniform distribution do the math.

PS No, but if you ask enough mathematicians 1/20 will say yes. However, I'll accept the null with an $\alpha$ of 5%.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.