# Overall type I error when repeatedly testing accumulating data

I have a question about group sequential methods.

According to Wikipedia:

In a randomized trial with two treatment groups, classical group sequential testing is used in the following manner: If n subjects in each group are available, an interim analysis is conducted on the 2n subjects. The statistical analysis is performed to compare the two groups, and if the alternative hypothesis is accepted, the trial is terminated. Otherwise, the trial continues for another 2n subjects, with n subjects per group. The statistical analysis is performed again on the 4n subjects. If the alternative is accepted, then the trial is terminated. Otherwise, it continues with periodic evaluations until N sets of 2n subjects are available. At this point, the last statistical test is conducted, and the trial is discontinued

But by repeatedly testing accumulating data in this fashion, the type I error level is inflated...

If the samples were independent of one another, the overall type I error, $\alpha^{\star}$, would be

$\alpha^{\star} = 1 - (1 - \alpha)^k$

where $\alpha$ is the level of each test, and $k$ is the number of interim looks.

But the samples are not independent since they overlap. Assuming interim analyses are performed at equal information increments, it can be found that (slide 6)

Can you explain me how this table is obtained?

The context is a z-test with known standard deviation. The first test statistic $z_1$, suitably standardized, has a Normal(0,1) distribution with cdf $\Phi$. So does the second statistic $z_2$, but--because the first uses a subset of the data used for the second--the two statistics are correlated with correlation coefficient $\sqrt{1/2}$. Therefore $(z_1, z_2)$ has a binormal distribution. The probability of a type I error (under the null hypothesis) equals the probability that either (a) a type I error occurs in the first test or (b) a type I error does not occur in the first test but does occur in the second test. Let $c = \Phi^{-1}(1 - 0.05/2)$ be the critical value (for a two-sided test with nominal size $\alpha$ = 0.05). Then the chance of a type I error after two analyses equals the chance that $|z_1| > c$ or $|z_1| \le c$ and $|z_2| > c$. Numeric integration gives the value 0.0831178 for this probability, agreeing with the table. Subsequent values in the table are obtained with similar reasoning (and more complicated integrations).
• @Marco,The correlation is straightforward to calculate because the test statistic is so simple: it's a linear combination of normal variables. (This is because we assume the variance is known.) Alternatively, you can think of the second statistic as being a sum of two independent random variables: the first one, $z_1$, plus the change created by the additional data, $z_1 - z_2$. In more complicated cases the correlation might be quite difficult to calculate: that's one reason this somewhat idealized situation is used to motivate the sequential tests! – whuber Mar 2 '11 at 8:40