What does it mean if test was rejected less than in 5% cases? I was reading a paper and authors performed multiple tests for Poisson distribution on single vectors. Large dataset of $n$ quite long vectors was used. The test was rejected only $0.02 \cdot n$ times under 0.05 level of significance.
I can imagine non-uniform distribution of p-values and I understand what means 10% of rejections under 5% significance (test's assumptions are not satisfied), but this case is different. Does it mean that suggested test statistic is asymptotically less than theoretical and test is not correct?
(I found test that was used)
 A: If the null hypothesis is correct, the distribution of the p-value should be uniform, by definition. I have come across two cases in my career where the results looked "too consistent" with the null...out of 50+ tests, we have most p-values $\geq 0.9$! 
In cases such as this, I considered two possible explanations:


*

*Samples not truly independent

*Incorrect null distribution (if parametric)


Since p-values test for "extremeness" they are not very good at individually testing for an abundance of "closeness"....more than expected by chance.
However, first you need to see how often you'd get $0.02*n$ rejections if the base rejection rate were $0.05$. This is a binomial test.
It could also be bad samples (inconsistent samples) where they do not follow any consistent distribution...this will have unpredictable effects. 
In the particular example, the sample sizes were actually quite moderate ($5 \leq n \leq 20)$ an the number of Monte Carlo runs were decent at $10,000$. However, the tests under consideration all involved large-sample approximations of one form or another, so my bets are on my second bullet here...they are really deriving approximate p-vaules.
