11
$\begingroup$

I'm in an introductory statistics class in which the probability density function for continuous random variables has been defined as $P\left\{X\in B\right\}=\int_B f\left(x\right)dx$. I understand that the integral of $\int\limits_a^af(x)dx=0$ but I can't rectify this with my intuition of a continuous random variable. Say X is the random variable equal to the number of minutes from time t that the train arrives. How do I calculate the probability that the train arrives exactly 5 minutes from now? How can this probability be zero? Is it not possible? What if the train does arrive exactly 5 minutes from now, how could it occur if it had probability 0?

Thanks.

|cite|improve this question
$\endgroup$
  • 2
    $\begingroup$ Standing some of these questions on their head is helpful. E.g., if your intuition says every possible time must have some strictly positive probability, then--because there is an uncountable set of possible times in any interval--your intuition implies the total probability is infinite. Obviously that intuition is wrong. One thing that has to be given up is the idea that a probability of zero implies an impossibility: that is not true. Similarly, a probability of one does not imply a certainty. $\endgroup$ – whuber♦ Nov 6 '13 at 4:17
  • $\begingroup$ @whuber That's what I can't rectify. If the probability of an event taking place is 0, then it should never happen. For example, if I have a standard six-sided die, the probability I roll any number $\mathcal{Z}\setminus\left\{1,2,3,4,5,6\right\}$ is 0 and therefore will never happen. Moreover, how can an event with probability 1 not be a certainty in the subsequent experiment? Could you provide an example? $\endgroup$ – geofflittle Nov 7 '13 at 0:04
  • 1
    $\begingroup$ Suppose you see a circle in which a chord is shown and it appears to be a diameter, prompting you to wonder "what was the chance that a randomly selected chord would not have been a diameter?" When the chord is obtained by choosing a pair of points uniformly and independently along the circumference, the answer is $1$, but this event did not occur. That provides (pretty strong!) evidence that the chord was not the result of the random process that you posited. One lesson afforded by such thought experiments is that intuitions based on finite probability spaces do not always generalize. $\endgroup$ – whuber♦ Nov 7 '13 at 18:00
8
$\begingroup$

You may be falling into the trap of regarding 'five minutes from now' as lasting some finite period of time (which would have a nonzero probability).

"Five minutes from now" in the continuous variable sense is truly instantaneous.

Imagine that the arrival of the next train is uniformly distributed between 8:00 and 8:15. Further imagine we define the arrival of a train as occurring at the instant the front of the train passes a particular point on the station (perhaps the midpoint of the platform if there's no better landmark). Consider the following sequence of probabilities:

a) the probability a train arrives between 8:05 and 8:10

b) the probability a train arrives between 8:05 and 8:06

c) the probability a train arrives between 8:05:00 and 8:05:01

d) the probability a train arrives between 8:05:00 and 8:05:00.01 (i.e. in the space of one hundredth of a second

e) the probability a train arrives between 8:05 and one billionth of a second later

f) the probability a train arrives between 8:05 and one quadrillionth of a second later

... and so on

The probability that it arrives precisely at 8:05 is the limiting value of a sequence of probabilities like that. The probability is smaller than every $\epsilon>0$.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ I understand this, but, assuming the train arrives, it arrives at some point in time. Why can't this limit still converge to some probability? $\endgroup$ – geofflittle Nov 7 '13 at 0:06
  • $\begingroup$ If you understand it, as you say, you can compute the probability in the indicated fashion. Let me make it easier: Imagine for convenience of calculation that the exact time that a train "arrives" (however we define it, as long as it's actually continuous) at a uniformly distributed time on the interval (0,1) (at whatever is a convenient time unit). What's the probability that the train arrives before time $x$, for some $x$ inside the interval? What's the probability that it arrives after time $x$? What's the probability that it arrives between $x$ and $x+dx$? ...(ctd) $\endgroup$ – Glen_b Nov 7 '13 at 0:59
  • $\begingroup$ (ctd)... To say it 'arrives at time $x$' for a continuous variable, means "what is the limit of that last probability as $dx\to 0?$. So, what is that limit? Work it out! That is the probability it converges to. This feature is intimately related to what makes a continuous pdf continuous. $\endgroup$ – Glen_b Nov 7 '13 at 0:59
  • $\begingroup$ Further note that if that last limit is anything but zero, your three probabilities (before $x$, after $x$ and "at" $x$) won't add to 1. $\endgroup$ – Glen_b Nov 7 '13 at 1:05
5
$\begingroup$

What if the train does arrive exactly 5 minutes from now, how could it occur if it had probability 0?

A probabilistic statement is not a statement about the possibility/feasibility of an event. It only reflects our attempt to quantify our uncertainty about it happening. So when a phenomenon is continuous (or is modeled as one), then our tools and current state of knowledge do not permit us to make a probabilistic statement about it taking a specific value. We can only make such a statement related to a range of values. Of course the usual trick here is to discretize the support, to consider"small" intervals of values rather than single values. Since continuous random variables bring great benefits and flexibility compared to discrete random variables, this has been found to be a rather small price to pay, perhaps as small as the intervals we are forced to consider.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ These statements are puzzling, perhaps because they could be interpreted in so many different ways. In some places you seem to deny the validity of the use of continuous distributions to model phenomena--and make a sharp distinction between the phenomenon and the model--and in other places you seem to drop that distinction altogether. My reading of it, which I suspect was not intended, is that you contend that the mathematical fact that $\Pr(X=a)=0$ for any continuous RV $X$ is in reality always false, but that makes it seem like you are denying the applicability of probability theory! $\endgroup$ – whuber♦ Nov 6 '13 at 4:14
  • 2
    $\begingroup$ Hi @whuber. Regarding the distinction between model and phenomenon, a map of the earth is not earth, but it may help you roam the earth. This is how I think about models, when I don't treat them as objects of pure intellectual pleasure (which they also are). As for the "zero probability" issue, it is an imperfection -after all, wouldn't it be great to have all the benefits of continuity and be able to make a probability statement about a single value? But being imperfect does not render something inapplicable of course, and as I write, this imperfection has proven of little importance. $\endgroup$ – Alecos Papadopoulos Nov 6 '13 at 10:09
  • $\begingroup$ You implicitly suppose that probability is some objective thing "out there" in your mapping analogy, but it's not. Probability has a meaning only within a model. I see no "imperfections" in the axioms of probability and indeed, one can make accurate, consistent statements about probabilities of single values: often they are zero. $\endgroup$ – whuber♦ Nov 6 '13 at 14:09
  • 2
    $\begingroup$ @whuber No I don't suppose that, and I don't understand where did you see that in what I wrote. I said "the map is not the earth", which means "what is in the model does not exist in reality", so how can you infer from that the exact opposite? The "imperfection" does not refer to the axioms of probability, but to what tools these axioms lead us to, and to how effectively these tools can be put to use in order to model, study and understand the real world. And it is obvious that I believe that probability is an effective tool. $\endgroup$ – Alecos Papadopoulos Nov 6 '13 at 14:58
4
$\begingroup$

To give you some intuition for the above, try the following (thought) experiment:

Draw a real line around zero with a ruler. Now take a sharp dart and let it fall from above randomly on the line(let's assume you will always hit the line and only the lateral positioning matters for the sake of the argument).

However many times you let the dart fall randomly on the line, you will never hit the point zero. Why? Think what is the point zero, think what is its width. And after you recognise that its width is 0, do you still think you can hit it?

Will you be able to hit point 1, or -2? Or any other point you pick on the line for that matter?

To get back to maths, this is the difference between the physical world, and a mathematical concept such as real numbers (represented by the real line in my example). Probability theory has quite a bit more complicated definition of probability than you will see in your lecture. To quantify the probability of events and any combination of their outcomes, you need a probability measure. Both the Borel measure and Lebesgue measure are defined for an interval [a, b] on the real line as: $$\mu([a,b])=b-a$$ from this definition you can see what happens with the probability if you reduce the interval to a number (setting a = b).

The bottom line is that based on our current definition of probability theory (dating back to Kolmogorov) the fact that an event has 0 probability does not mean it cannot occur.

And as far as your example with the train goes, if you will have an infinitely precise watch, your train will never arrive exactly on time.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ You say "you will never hit the point zero", but what can you say of the point that I hit in my first dart throw? Let $x$ be the point that I hit. Before throwing my dart, you would have said "you will never hit the point $x$", but I've just hit it. Now what? $\endgroup$ – geofflittle Nov 7 '13 at 0:09
  • $\begingroup$ I think that you have to differentiate between the question: What is the probability that I will hit some point? If we agree that you always throw a dart and it always hits somewhere along the line, that probability is 1. Also, I am not just saying you won't hit 0. I am saying that the probability that you hit ANY point you pick BEFORE throwing the dart is 0. In fact you can pick any finite set of points and the probability will still be 0. $\endgroup$ – means-to-meaning Nov 7 '13 at 1:22
  • $\begingroup$ Regarding your question, I get your point, but to ask about probabilities of events after their occurrence is non-sensical. A statement such as P(X=x) refers to the future realisation of a random variable X. So AFTER you hit some point, I won't be saying anything about it. (large caps used just to point out the time-flow, not to shout…) $\endgroup$ – means-to-meaning Nov 7 '13 at 1:22
1
$\begingroup$

A probability distribution has to have an area of unity. If the measure is continuous then there is an infinite number of values that it can take (i.e. an infinite number of values along the x-axis of the distribution). The only way that the total area of the probability distribution can be finite is for the value at each of the infinite number of values to be zero. One divided by infinity.

In 'real life' there can be no measures that take an infinite number of values (by several different philosophical arguments that don't matter much here) so no value need take a probability of exactly zero. A useful practical argument is based on the finite precision of real-world measurements. If you use a stopwatch that measures to one tenth of a second, the train will have one tenth of a second in which to arrive in 'exactly' five minutes.

|cite|improve this answer
$\endgroup$
  • 2
    $\begingroup$ The first paragraph here provides some vague intuition, though the deductive steps are incorrect. There are plenty of distributions that admit an infinite number of values yet each value has strictly positive probability. The second paragraph might profit from a rewording that emphasizes that to each measurement value is associated a (small) interval of possible values of the underlying quantity of interest. $\endgroup$ – cardinal Nov 6 '13 at 5:46
  • $\begingroup$ What is the difference between a strictly positive value (of a finite value divided by infinity?) and zero in this context? $\endgroup$ – Michael Lew Nov 6 '13 at 5:56
  • 1
    $\begingroup$ My point, probably poorly made, is that the argument in the first paragraph is based on the false premise that because the random variable can take on infinitely many values, each individual outcome must have probability zero. This is, of course, incorrect (Poisson, geometric, etc.); the concept of "infinity" is not a strong enough one here, we require uncountability. $\endgroup$ – cardinal Nov 6 '13 at 14:31
0
$\begingroup$

Other people have answered why the probability is zero (if you approximate time as being continuous, which it is effectively not, but anyway...) so I will just echo it briefly. To answer the last question that the OP asked---"how could it occur if it had probability 0?"---lots and lots of things can occur if they have probability zero. All a set of probability zero $A$ means is that, in the space of possible things that could happen, the set $A$ takes up no space. That is all. It is not more meaningful than this.

I am writing this to hopefully address something else that the OP said in the comments:

You say "you will never hit the point zero", but what can you say of the point that I hit in my first dart throw? Let 𝑥 be the point that I hit. Before throwing my dart, you would have said "you will never hit the point 𝑥", but I've just hit it. Now what?

This is a very good question and one that, when I began to learn about probability, I struggled with. Here is the answer: it isn't equivalent to the question that you originally asked! What you have done is bring time into the analysis, and that means that the underlying probability structure changes to become much more intricate. Here is what you need to know. A probability space $(\Omega, A, \mu)$ consists of three things: an underlying space $\Omega$, such as $\mathbb{R}$ or $\mathbb{Z}$; a set of all possible outcomes on this space, such as the set of all half-open intervals on $\mathbb{R}$, and a measure $\mu$ that satisfies $\mu(\Omega) = 1$. Your original problem lives in the space $([a,b], \text{all half open intervals on $[a,b]$}, \nu)$ where $\nu$ is Lebesgue measure (this means that $\nu( [c,d) ) = \frac{1}{d - c}$). In this space, the probability that you hit any single point $x \in [a,b]$ is zero for the reasons discussed above---I think we have this cleared up. But now, when you say things like the quoted passage above, you are defining something called a filtration, which we will write as $\mathcal{F} = \{\mathcal{F}_t\}_{t \geq 0}$. A filtration in general is a collection of subsets of $A$ that satisfy $\mathcal{F}_t \subseteq \mathcal{F}_s$ for all $t < s$. In your case, we can define the filtration $$ \mathcal{F}_t = \{x \in [a,b]: \text{dart hit $x$ at time $t' < t$} \}. $$ Now, in this new subset of your outcome space, guess what---you're right! You have hit it and, after your first throw, your probability of having hit that point when restricted to the filtration $\mathcal{F}_1$ is 1.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Because you are using technical language, it would be best to employ standard meanings for the terms. In particular, what you call an "outcome" is usually termed a (basic) event: the outcomes are the elements of $\Omega.$ Your formula for (normalized) Lebesgue measure is incorrect: I suspect you intended $\nu([c,d])=(d-c)/(b-a).$ On a more fundamental level, it isn't clear why you need to invoke the machinery of stochastic processes in order to discuss a random variable modeling the time of a single event, nor is it evident this provides any insight. $\endgroup$ – whuber♦ Apr 23 at 21:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.