Overall type I error when repeatedly testing accumulating data

Question

I have a question about group sequential methods.

According to Wikipedia:

In a randomized trial with two treatment groups, classical group sequential testing is used in the following manner: If n subjects in each group are available, an interim analysis is conducted on the 2n subjects. The statistical analysis is performed to compare the two groups, and if the alternative hypothesis is accepted, the trial is terminated. Otherwise, the trial continues for another 2n subjects, with n subjects per group. The statistical analysis is performed again on the 4n subjects. If the alternative is accepted, then the trial is terminated. Otherwise, it continues with periodic evaluations until N sets of 2n subjects are available. At this point, the last statistical test is conducted, and the trial is discontinued

But by repeatedly testing accumulating data in this fashion, the type I error level is inflated...

If the samples were independent of one another, the overall type I error, $\alpha^{\star}$, would be

$\alpha^{\star} = 1 - (1 - \alpha)^k$

where $\alpha$ is the level of each test, and $k$ is the number of interim looks.

But the samples are not independent since they overlap. Assuming interim analyses are performed at equal information increments, it can be found that (slide 6)

Can you explain me how this table is obtained?

Answer 1

The following slides, through 14, explain the idea. The point, as you note, is that the sequence of statistics is correlated.

The context is a z-test with known standard deviation. The first test statistic $z_1$, suitably standardized, has a Normal(0,1) distribution with cdf $\Phi$. So does the second statistic $z_2$, but--because the first uses a subset of the data used for the second--the two statistics are correlated with correlation coefficient $\sqrt{1/2}$. Therefore $(z_1, z_2)$ has a binormal distribution. The probability of a type I error (under the null hypothesis) equals the probability that either (a) a type I error occurs in the first test or (b) a type I error does not occur in the first test but does occur in the second test. Let $c = \Phi^{-1}(1 - 0.05/2)$ be the critical value (for a two-sided test with nominal size $\alpha$ = 0.05). Then the chance of a type I error after two analyses equals the chance that $|z_1| > c$ or $|z_1| \le c$ and $|z_2| > c$. Numeric integration gives the value 0.0831178 for this probability, agreeing with the table. Subsequent values in the table are obtained with similar reasoning (and more complicated integrations).

This graphic depicts the binormal pdf and the region of integration (solid surface).