The p-postulate is the notion that equal p-values provide equal evidence against the null hypothesis.
Wagenmakers et al (2008) write:
If p-values truly reflect evidence, a minimum requirement is that equal p-values provide equal evidence against the null hypothesis (i.e., the p-postulate). According to the p-postulate, p = .05 with 10 observations constitutes just as much evidence against the null hypothesis as does p = .05 after 50 observations.
They cite Royall (1986) as the source for their definition of the p-postulate. They also go on saying that this postulate is false. Is it?
- Royall, R. N. (1986). The Effect of Sample Size on the Meaning of Significance Tests, The American Statistician, 40:4, 313-315
- Wagenmakers, E.-J., Lee, M. D., Lodewyckx, T., & Iverson, G. (2008). Bayesian versus frequentist inference. In H. Hoijtink, I. Klugkist, and P. A. Boelen (Eds.), Bayesian Evaluation of Informative Hypotheses, pp. 181-207. Springer: New York.