# Can a statistical test return a p-value of zero?

I don't mean a value close to zero (rounded to zero by some statistical software) but rather a value of literally zero. If so, would it mean that the probability of getting the obtained data assuming the null hypothesis is true is also zero? What are (some examples) of statistical tests that can return results of this sort?

Edited the second sentence to remove the phrase "the probability of the null hypothesis".

-
You might find the examples shown in the closely related question at stats.stackexchange.com/questions/90325/… to be helpful. – whuber Mar 18 '14 at 20:15

It will be the case that if you observed a sample that's impossible under the null (and if the statistic is able to detect that), you can get a p-value of exactly zero.

That can happen in real world problems. For example, if you do an Anderson-Darling test of goodness of fit of data to a standard uniform with some data outside that range - e.g. where your sample is (0.430, 0.712, 0.885, 1.08) - the p-value is actually zero (but a Kolmogorov-Smirnov test by contrast would give a p-value that isn't zero, even though we can rule it out by inspection).

Likelihood ratio tests will likewise give a p-value of zero if the sample is not possible under the null.

As whuber mentioned in comments, hypothesis tests don't evaluate the probability of the null hypothesis (or the alternative).

We don't (can't, really) talk about the probability of the null being true in that framework (we can do it explicitly in a Bayesian framework, though -- but then we cast the decision problem somewhat differently from the outset).

-
In the standard hypothesis testing framework there is no meaning to "the probability of the null hypothesis." We know that you know that but it looks like the OP doesn't. – whuber Mar 18 '14 at 20:47
Perhaps explicating this a bit: The standard uniform includes only values from 0 to 1. Thus, a value of 1.08 is impossible. But this is really rather odd; is there a situation where we would think that a continuous variable is distributed uniformly, but not know its maximum? And if we knew its maximum was 1, then 1.08 would just be a sign of a data entry error. – Peter Flom Mar 18 '14 at 21:05
@whuber Does it work if I rephrase to "If so, would it mean that null hypothesis is definitely false"? – user1205901 Mar 18 '14 at 21:09
@whuber Okay, thanks, I can certainly do that, and I'll get rid of my rambling comments as well. I'm not thinking clearly this morning ... in respect of your last sentence, can you give me a hint about what sort of circumstances that comes up in? – Glen_b Mar 18 '14 at 23:00
@whuber I'd also be interested in which circumstances a true $H_0$ can have a (true) zero p. I think that's very relevant to this question here, but it might be sufficiently different to be worth asking as a question in its own right. – Silverfish Feb 4 '15 at 15:20

In R, the binomial test gives a P value of 'TRUE' presumably 0, if all trials succeed and hypothesis is 100% success, even if number of trials is just 1:

> binom.test(100,100,1)

Exact binomial test

data:  100 and 100
number of successes = 100, number of trials = 100, p-value = TRUE   <<<< NOTE
alternative hypothesis: true probability of success is not equal to 1
95 percent confidence interval:
0.9637833 1.0000000
sample estimates:
probability of success
1

>
>
> binom.test(1,1,1)

Exact binomial test

data:  1 and 1
number of successes = 1, number of trials = 1, p-value = TRUE   <<<< NOTE
alternative hypothesis: true probability of success is not equal to 1
95 percent confidence interval:
0.025 1.000
sample estimates:
probability of success
1

-
That's interesting. Looking at the code, if p==1 the value computed for PVAL is (x==n). It does a similar trick when p==0, giving (x==0) for PVAL. – Glen_b Mar 27 '15 at 2:04
However, if I put in x=1,n=2,p=1, it doesn't return FALSE, but the smallest p-value it can return, so it doesn't get to that point in the code in that case (similarly with x=1,n=1,p=0). So it seems as if that piece of code perhaps will only be run when it's going to return TRUE. – Glen_b Mar 27 '15 at 2:10