3
$\begingroup$

It violates some intuition in me that no matter what particular outcome has occurred of a variable described by a continuous distribution, it must have had probability 0.

In Frequentist inference, I feel ok (but not great) by recognizing that the asymptotic value of the number of occurrences of any outcome divided by n (number of trials) is 0 as n approaches infinity.

In Bayesian, I don't feel satisfied (yet) with just saying that you can (correctly) have had 0 expectation of certainty for an outcome that has indeed happened.

I'm just hoping to find a perspective to understand this better.

Thanks!

Edit: Due to some confusion in the comments, I just want to emphasize that this is not a question about why point events must always have probability 0 in a (pure) continuous distribution. That is a strictly mathematical consequence because, for a given probability space, there cannot exist an uncountable number of disjoint events with probability greater than 0.

I'm just having trouble reconciling how we can interpret that with a Bayesian approach.

Improve this question
$\endgroup$
4
  • $\begingroup$ Your intuition, to be consistent, must then insist that the number of possible outcomes can be at most countable. That would rule out the (direct) application of most of the tools of probability and statistics, including the Normal distribution. $\endgroup$
    – whuber ♦
    Commented Dec 14, 2013 at 0:32
  • $\begingroup$ I'm fully aware that there is no other sensible way to define continuous distributions, and I'm also aware that continuous distributions are useful and would prefer not to lose them. What I'm having trouble with is reconciling what I understand to be the interpretation of Bayesian Inference with (for example) a continuous distribution. $\endgroup$
    – James Sydow
    Commented Dec 14, 2013 at 0:39
  • $\begingroup$ My comment has nothing to do with distributions nor with continuity of distributions nor with distributions at all. (For instance, a random variable with an uncountable range need not be continuous.) Since you are aware of some of the mathematical implications, where is the difficulty with the interpretation? $\endgroup$
    – whuber ♦
    Commented Dec 14, 2013 at 4:10
  • $\begingroup$ I believe a couple of people deleted their comments, so it looks like I'm responding to you directly in my question edit. Anyway, to simplify my motivation, I find it a confusing thing for reasons outlined above. I'm not for one moment trying to fight it; I'm just trying to understand it. It seems to me a most natural (even if wrong) response to think probability = 0 -> impossible. $\endgroup$
    – James Sydow
    Commented Dec 16, 2013 at 3:25

1 Answer 1

Reset to default
4
$\begingroup$

Putting aside the philosophical waste-of-time that is the contemplation of whether or not the universe is continuous, the important thing is that there is nothing that is measurable as continuous. Any observation method has intrinsic errors; any observation method has finite resolution.

Therefore, any apparently continuous measurand must in reality be described in terms of an interval that is larger than a single point. And, for a continuous pdf, the pdf over the interval will integrate to more than zero, as long the pdf is positive over at least some sub-interval of it.

If the posterior pdf is zero over the whole interval that represents the measured value, then either the prior is wrong, or the likelihood is wrong (or they're both wrong).

And, to go a little further, and only half tongue-in-cheek, if you have observed something inexplicable, it's much more likely to be the result of something you thought to be impossible, rather than something you knew to be improbable. (with a nod to Douglas Adams's creation Dirk Gently, who was, to my knowledge, the first to express that idea, though not in those words)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.