7
$\begingroup$

For example, suppose I am told that 10 data points come IID from a normal distribution with some mean and variance. Isn't the probability of realizing each of these values zero? Shouldn't the fact that the probability of drawing each data point being zero imply that the likelihood is zero? Why can I sample particular values rather than being forced to sample intervals, for example?

I understand that simulating draws from continuous random variables with a computer is a useful fiction, since no computer has infinite precision. However, sometimes problems are posed such that data points actually come from a continuous distribution, rather than a discrete approximation to a continuous distribution.

This seems logically impossible, or at least the zero probability should be reflected in the likelihood calculations. There is much commentary in intro probability courses about continuous RVs taking scalar values with zero probability but then this is never mentioned in a statistics class when you are told that data is IID from a continuous distribution.

I know the question is simple but I haven't seen a satisfactory answer anywhere.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ +1 One interesting thought that comes from reflecting on this question is the possibility that any real number actually can be drawn via a computer algorithm that almost surely terminates. It requires that we take a pragmatic point of view: namely, that after drawing the number, it is required only that the computer be able to discover enough significant digits to be able to report its results to any desired finite accuracy. The likelihood issue is addressed by noting that the likelihood function implicitly contains a differential element that is canceled in any likelihood ratio. $\endgroup$
    – whuber ♦
    Commented Dec 28, 2015 at 21:48

1 Answer 1

Reset to default
2
$\begingroup$

It's because zero probability should not be conflated with impossibility. Of course some value has to be sampled, so rather than observing that number and saying to yourself "what was the probability I would have observed this?" and then being confounded by the answer, pick a number arbitrarily and then draw samples until you get that number. It will never happen. That better illustrates the idea.

By the way, this is purely thought experiment since as far as I can tell it's not actually possible to sample from a continuous distribution, which is really just a mathematical abstraction. (Consider for instance how many supposedly uniform$(0, 1)$ random numbers ever generated have been irrational, an event that is supposed to have probability one.)

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ If you want irrational values with probability $1$, just multiply all values returned by your uniform pseudo RNG by $1 - 1/2^{2000}$ and add $1/(\pi 2^{2000})$ :-). More seriously, it's relatively easy to write a program to generate uniform pseudo random real numbers in (0,1) (with infinite precision), provided they never have to be tested for exact equality--which is unnecessary, since any such test has zero probability of being true! One (inefficient) way is to generate each successive binary digit independently until the number is known sufficiently precisely for the calculations. $\endgroup$
    – whuber ♦
    Commented Dec 28, 2015 at 21:56
  • $\begingroup$ @whuber True, I was kind of thinking that the number needs to be stored or represented somehow by the computer, and if that representation were explicit it would have to be in the form of a rational number. But maybe as you suggest it could be represented "implicitly" with a finite number of digits along with a method for generating the rest. $\endgroup$
    – dsaxton
    Commented Dec 28, 2015 at 23:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.