I have created a vector of length 400 with 6 unique values of relatively equal proportions (1-6). I have drawn two random samples from the vector, each $n=26$, 10,000 times, performed $t$-tests, added the $p$-values to a list and output a histogram of the $p$-value frequency. The histogram of the $p$-values shows a cyclical clumping effect for $p$-values greater than $0.7$. I would like to know why this occurs.

Here is my R-code:


pval.list <-vector("list", length(x))
alpha <- 0.05
conf.level <- 1 - alpha
for(i in 1:x){

  s1 <- sample(landscape, size = sample.size, replace = TRUE, prob = NULL)
  s2 <- sample(landscape, size = sample.size, replace = TRUE, prob = NULL)
  result.t.test <- t.test(s1,s2, conf.level=conf.level)
  pval.list[i] <- result.t.test["p.value"] 
hist(pvalue, main = paste("Histogram of" , "p-value"), breaks= 100)

Histogram of p-values


The effect you're seeing is due to the small number of distinct values of your underlying population combined with your smallish sample size. In order to get a large value for your t-statistic (and therefore a high p-value) with a sample size of 26, you need to have a fairly extreme sample, but there aren't all that many ways to construct an extreme sample with this population, so you don't have all that many different large p-values coming out of the analysis - and consequently you have noticeably large gaps between them.

To confirm this, try running the same code with different values for sample.size:

enter image description here

enter image description here

As you can see, with a sample size of 100, the right tail of the graph has filled in a lot, and with a sample size of 1000, you're getting pretty close to a uniform distribution.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.