Suppose we have a $2\times 2$ table of observations $X_0=(a_0,b_0,c_0,d_0)$ with given sample sizes $n_1=a_0+c_0$ and $n_2=b_0+d_0$. We want to test the hypothesis that the probabilities of success in both samples, $p_1$ and $p_2$, are equal given the usual assumptions. Barnard's test works, first, by choosing a statistic $T$ which outputs a real number for any table $X=(a,b,c,d)\in\mathbb{N}^4$ such that $a+c=n_1$ and $b+d=n_2$. Then, assuming $p_1=p_2=\pi$, one computes the probability that a table $X$ will occur for every $T(X)\geq T(X_0)$ and sums these probabilities to obtain the exact probability, $p(\pi)$, of getting a table at least as "extreme" as $X_0$. Finally, the $p$-value is taken to be the supremum $\sup_{\pi\in(0,1)}p(\pi)$.

In every setting I have seen this test used, the test statistic was chosen to be what Mehta and Senchaudhuri refer to as the Wald statistic or its absolute value, i.e.


where $\hat{p}_1=\frac{a}{n_1}$, $\hat{p}_2=\frac{b}{n_2}$, and $p_0=\frac{a+b}{n_1+n_2}$. But this is unsettling to me. The calculation of $p(\pi)$ is performed using the assumption that $p_1=p_2=\pi$, in which case we know the exact distribution of $\hat{p}_1-\hat{p}_2$. Using independence, the variance of this distribution is $\pi(1-\pi)\left(\frac{1}{n_1}+\frac{1}{n_2}\right)$, so why would we rescale by an approximation to the standard deviation rather than the exact standard deviation of the distribution of $\hat{p}_1-\hat{p}_2$? That is to say, wouldn't $\frac{\vert\hat{p}_1-\hat{p}_2\vert}{\sqrt{\pi(1-\pi)\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}}$ be a more logical choice of test statistic than the Wald statistic?

I should note that, because $\pi(1-\pi)\left(\frac{1}{n_1}+\frac{1}{n_2}\right)$ does not depend on $X$, using $\frac{\vert\hat{p}_1-\hat{p}_2\vert}{\sqrt{\pi(1-\pi)\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}}$ as the test statistic is equivalent to just using $\vert\hat{p}_1-\hat{p}_2\vert$ which is perhaps more computationally efficient. So I am lead to wonder why the absolute difference was not simply used instead of the Wald statistic. Does using the Wald statistic give the test greater power than the absolute difference, or is there perhaps some other reason?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.