0
$\begingroup$

This question already has an answer here:

I was wondering why the expected value (i.e., long-run average) of the p-values from a simulation of a binomial experiment in the below experiment becomes roughly about $.61$, regardless of $p$; the probability of success?

Here is my simulation code in R:

simulation <- function(n, p, n.sim){
    fun = function(){
      x = rbinom(1, n, p)
     pe = x/n
p.value = binom.test(x, n, p)[[3]]
    c(p.value, pe)
}
sim <- t(replicate(n.sim, fun()))
par(mfcol = c(2, 2))
plot(sim[, 1], 1:n.sim, xlim = c(0, 1), pch = 19, col = 2, main = "p.value")
plot(sim[, 2], 1:n.sim, xlim = c(0, 1), pch = 19, col = 4, main = "proportion")
abline(v = p, lty = 2, col = 2)
hist(sim[, 1], main = "p.value")
hist(sim[, 2], main = "proportion")
list(p.value = mean(sim[, 1]), proportion = mean(sim[, 2]))
}
simulation(n = 15, p = .1, n.sim = 1e3) ## Change p to whatever and p.value remains around .61

enter image description here

$\endgroup$

marked as duplicate by Kodiologist, Michael R. Chernick, mdewey, kjetil b halvorsen, Nick Cox Aug 20 '17 at 5:54

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 1
    $\begingroup$ Since p-values are uniformly distributed under the null (at least approximately, depending on the circumstances and the test), and uniform variables have a mean of $0.5$, it sounds like your computer experiments are in line with theory. $\endgroup$ – whuber Aug 18 '17 at 20:40
  • $\begingroup$ @whuber, but change the sample size (I should say n to 15) and then what happens? $\endgroup$ – rnorouzian Aug 18 '17 at 20:45
  • $\begingroup$ It will make no material difference. Hypothesis tests are constructed so that p-values will have uniform distributions under the null hypothesis, or as close to uniform as is feasible, regardless of sample size and regardless of the alternative. (A "composite" null creates a complication that I am glossing over here, because it's not a conceptual one.) $\endgroup$ – whuber Aug 18 '17 at 20:54
  • 1
    $\begingroup$ Not at all: it has been explained many times on this site. $\endgroup$ – whuber Aug 18 '17 at 21:26
  • 1
    $\begingroup$ The case of a simple null but discrete test statistic is also discussed here $\endgroup$ – Glen_b Aug 19 '17 at 1:06