# Not sure on how to interpret the p-value for a permutation test

I was running a permutation test on my data. My goal was to test difference in means in two samples, by resampling one of them.

I got results like these: And From my understanding, the p-value here is the proportion of data points in the sample that have bigger mean difference in the permutation groups compared to the complete data. Thus, a p-value closer to 1 tells me that 100% of the data in the permuted samples have bigger mean difference, and the p-value of 0.02 tells me that only 2% of the data in the permuted samples have bigger mean difference. In the second I reject the null hypothesis, but in the first case I do not. My question is: why?

Why 90%, 95%, 99%... of data in the permuted samples having bigger difference from the actual mean difference does not tell me to reject the null hypothesis? I mean, the permuted groups seems to have really significant differences in means than the actual one for the whole data, but I do not reject the null hypothesis. When the p-value is as low as 0.05, I can visualize that there are a lot of difference in the permuted samples comparing to the actual mean. But in only one case I reject the null hypothesis. This is quite confusing.

• Are you sure the p-value in the first plot is not just a display problem? e.g. 1.00E-3 truncated badly could be displayed as 1.00
– mkt
Aug 2, 2022 at 13:41
• Another possibility is that you are doing a one-sided test
– mkt
Aug 2, 2022 at 13:42
• Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
– Community Bot
Aug 2, 2022 at 14:18
• It seems like the two figures are using different data sets. Is that correct? Why is it confusing that the p-value would be different for two different datasets? Aug 2, 2022 at 14:56
• both figures are using different datasets but my problem is to interpret the p-value: why a p close to 1 does not reject the null hypothesis and a p close to 0 does in a permutation test?Both of them are telling me that the difference in means in the permutation group are very different to the difference in means in the whole data,one of them tells me that something close to 100% are greater in the permutation group, other tells me that something close to 0% are less in the permutation group Aug 2, 2022 at 15:10