What is the difference between nominal significance and genome wide significance?

Question

I need to determine a reasonable sample size for my Inifinium 450k methylation array case-control experiment and wondered why epigenome-wide association scan (EWAS) studies typically use a P value threshold of $1 \cdot 10^{-6}$. Can I use a nominal significance $P<0.05$ instead?

Answer 1

No, you absolutely cannot. There is a phenomena in hypothesis testing called the multiple comparisons problem. This is hugely exacerbated in genetic studies due to the high number of hypotheses being conducted at once.

As a proof of concept, let's examine a simple hypothesis.

$H_0: \mu = 0$

$H_1: \mu \ne 0$

In this case, we might do a $Z$ test to obtain a $Z$ statistic to test our $H_0$. If we assume that $H_0$ really is true (i.e., $\mu =0$), then there is a 95% probability that our test statistic will lie in the null region.

This is good, because we will probably fail to reject $H_0$. However, there is a fundamental rule in probability such that

$$P(A \cap B) = P(A)P(B|A)$$

and when $A$ and $B$ are independent, then $P(B|A)=P(B)$

$$P(A \cap B)=P(A)P(B)$$

Thus if you have now two true $H_0$ hypotheses, each with a 95% probability of truly appearing in the null region. If the two tests are independent, the probability of both appearing in the null region is

$$0.95^2$$

and for $n$ independent tests,

$$0.95^n$$

This quickly almost guarantees false positives, as true nulls will appear as $H_1$ just due to multiplicity. Thus you can see why testing multiple hypotheses at the same time is troublesome. To correct for this, we have to adjust the acceptance threshold ($\alpha$) so that we do not get false positives (type 1 error). As a note, completely independent tests are not common in genetics, and this is a simplified situation.

There are lots of methods of doing this, but you can start by looking into bonferronni correction (the simplest method, though often misused) which corrects $\alpha$ for even one false discovery, and false discovery rate control which effectively caps the false discovery rate at a certain level (usually 5%).

To read more about how the P value of $5 \cdot 10^{-8}$ started being used in genetics, see this paper. I have never heard of an epigenome wide significance level, though it is probably an adaptation of the genome wide significance level for a smaller number of tests.