# An exact paired perm.test is returning a p-value of 0 for large sample sizes

I'm trying to find the p-value from an exact paired permutation test. For small samples it works fine:

> library(exactRankTests)
> n <- 1000
> perm.test(runif(n, 0.9, 1),
runif(n, 0.8, 1),
paired=TRUE, exact=TRUE)$p.value [1] 2.907099e-116  but for bigger samples it gives exactly 0: > n <- 1500 > perm.test(runif(n, 0.9, 1), runif(n, 0.8, 1), paired=TRUE, exact=TRUE)$p.value
[1] 0


How can I obtain the exact value? And if I cannot, how should I report the p-value in a journal article?

-
The help on perm.test says that the first two arguments are expected to be integers, and describes a somewhat surprising set of steps when they aren't ... and even then, it looks like it assumes the smallest gap in values is less than tol (is it?). It then links to a description of how the p-values are computed (in the help of another function in the same package). People attempting to answer might need to be aware of this. If anyone is inclined to look at code, it's in perm.test.default in the abovementioned package. –  Glen_b Jul 31 at 23:38

## 2 Answers

You can just say that the p-value is approximately 0 or smaller than $10^{-100}$. For practical purposes the exact value of such a small p-value is irrelevant and might even be unreliable.

Take a look at the following discussion by Andrew Gelman on this topic:

http://andrewgelman.com/2013/01/25/extreem-p-values/

There is also a similar discussion in another question:

Why does R have a minimum p-value of $\le$ 2.22e-16? How should p be reported in such cases?

-

You are not actually getting a p-value of 0 - you're getting p-value that's been rounded to 0 because, as @Karri suggests from this question, it's smaller than the smallest double precision floating point number.

You need not report an exact p-value, indeed very small p-values, or p-values reported with large numbers of significant digits, are something of a bad practice, as they imply a level of precision in your data that isn't actually there. The easiest way is to pick something that most people would read as very significant, and report it as less than that number. For example, p < 0.001. There's not that much information to be had by reporting a smaller, more precise number.

-