Several topics are talking about this but in my case my p values are generated differently.

My p values will be calculated simulating $N$ (in this case 5000) random gene groups of size $g$ of the same size as the original gene group(pooling the genes from the total number of genes $G$, such that $g\subseteq G$ ) and checking how many of those groups gives higher statistics comparing to the value that the real group gives. Being $w$ the number of random cases that are greater than the observed value:

Then the p value will be: $$ p =\dfrac{(w+1)}{(N+1)}, $$ Adding a unitary summation to avoid p values equal to 0.

Since I did 5000 randomizations, my smallest p value will be $$ p =\dfrac{1}{5001}=0.00019996 , $$

Visualizing my p value distribution:

Since I am testing a data set that I know is not random, I obtained what I was expecting, a lot of significant p values.

However, my concerns start here. If I do fdr or BH I obtain a good corrected p value, but if I use hommel or bonferroni all my corrected p values become 1.

My p value calculation is similar to the one used in permutation testing, but I do not thing my case is exactly the same.

Here my doubts:

1) Does the correction of p values make sense in my analysis?

2) Are the FWER correction methods to conservative for my p values?

  • 1
    $\begingroup$ Bonferroni may be too strict a method for this many hypotheses. $\endgroup$ – user2974951 Sep 14 '18 at 10:05
  • 1
    $\begingroup$ Indeed it is, but maybe that happens because it makes no sense to correct for it... $\endgroup$ – Minus Sep 14 '18 at 10:09
  • $\begingroup$ I'm not really sure, but are you using permutation tests for determining the p-value? If so these do not need to be adjusted. $\endgroup$ – user2974951 Sep 14 '18 at 10:11
  • $\begingroup$ Thats the thing, not sure if it is really permutation test. Because I am not comparing a suffled dataset with an original one. I am just building a null distribution based on random sampled genes and then observing how many values are more extreme than the one I have in the real group. $\endgroup$ – Minus Sep 14 '18 at 10:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.