# P values all equal to 1 after using bonferroni or hommel

My p values will be calculated simulating $N$ (in this case 5000) random gene groups of size $g$ of the same size as the original gene group(pooling the genes from the total number of genes $G$, such that $g\subseteq G$ ) and checking how many of those groups gives higher statistics comparing to the value that the real group gives. Being $w$ the number of random cases that are greater than the observed value:

Then the p value will be: $$p =\dfrac{(w+1)}{(N+1)},$$ Adding a unitary summation to avoid p values equal to 0.

Since I did 5000 randomizations, my smallest p value will be $$p =\dfrac{1}{5001}=0.00019996 ,$$

Visualizing my p value distribution:

Since I am testing a data set that I know is not random, I obtained what I was expecting, a lot of significant p values.

However, my concerns start here. If I do fdr or BH I obtain a good corrected p value, but if I use hommel or bonferroni all my corrected p values become 1.

My p value calculation is similar to the one used in permutation testing, but I do not thing my case is exactly the same.

Here my doubts:

1) Does the correction of p values make sense in my analysis?

2) Are the FWER correction methods to conservative for my p values?

• Bonferroni may be too strict a method for this many hypotheses. – user2974951 Sep 14 '18 at 10:05
• Indeed it is, but maybe that happens because it makes no sense to correct for it... – Minus Sep 14 '18 at 10:09
• I'm not really sure, but are you using permutation tests for determining the p-value? If so these do not need to be adjusted. – user2974951 Sep 14 '18 at 10:11
• Thats the thing, not sure if it is really permutation test. Because I am not comparing a suffled dataset with an original one. I am just building a null distribution based on random sampled genes and then observing how many values are more extreme than the one I have in the real group. – Minus Sep 14 '18 at 10:16