1
$\begingroup$

When we perform a z-test for the difference in proportions between two groups of binary data, we use the fact that under the null hypothesis the proportions are assumed equal in order to estimate the SE. Suppose we have $\hat{p}_1$ and $\hat{p}_2$ as the sample means for the two groups. Then the test statistic we use under the null hypothesis is:

$z=\frac{\hat{p_1}-\hat{p_2}}{SE(\hat{p_1}-\hat{p_2})}$

where $SE(\hat{p_1}-\hat{p_2})=\sqrt{\hat{p}(1-\hat{p})(\frac{1}{n_1}+\frac{1}{n_2})}$

and $\hat{p}$ is the overall proportion.

In contrast, when we perform a standard t-test between two groups of continuous outcomes, A and B say, we use a pooled SD to estimate the SE:

$SE(\overline{X}_A-\overline{X}_B)=\sqrt{s_{pooled}^2(\frac{1}{n_1}+\frac{1}{n_2})}$

where $s_{pooled}^2=\frac{(n_A-1)s_A^2 + (n_B-1)s_B^2}{n_A+n_B-2}$

In the above, $s_A^2$ and $s_B^2$ are functions of $\overline{X}_A$ and $\overline{X}_B$ respectively. However, under the standard null hypothesis of $\mu_A=\mu_B=\mu$, so I'm wondering why we don't use the overall sample standard deviation $s$, which is a function of $\overline{X}$, instead of $s_p$. This would still gives us a test statistic that follows a t-distribution (under the null hypothesis), only now with $n_A+n_B-1$ degrees of freedom instead of $n_A+n_B-2$.

I had a couple of thoughts, but wasn't sure. Firstly, if we didn't use the equal-means assumption in the binary data case, then it would be difficult to derive a test statistic that was z-distributed. However, using the following SE estimate, for example:

$SE_{alternative}(\hat{p_1}-\hat{p_2})=\sqrt{\frac{\hat{p_1}(1-\hat{p_1})}{n_1}+\frac{\hat{p_2}(1-\hat{p_2})}{n_2}}$

would give us a t-distributed test statistic (although I'm not sure how many degrees of freedom - this is probably a similar scenario to Welch's t-test for continuous data).

Secondly, in the continuous data case, the test statistic using the $s_p$-based SE is presumably more powerful than the $s$-based one, although I'm not sure if this is the reason why we use the $s_p$-based one.

So in summary, why do we make use of the equal-means assumption in estimating the SE for risk differences but not mean differences?

$\endgroup$
  • $\begingroup$ This exact formula that uses $S_A^2$ and $S_B^2$ is derived from the likelihood ratio test for nested models, and you can't simply change it, even if it gives proper distribution under $H_0$ $\endgroup$ – Łukasz Grad Mar 9 '17 at 20:35
  • $\begingroup$ Do you have a reference where I can find this derivation? $\endgroup$ – NeB Mar 9 '17 at 22:48
  • $\begingroup$ webpages.cs.luc.edu/~jdg/w3teaching/stat_305/sp11/… $\endgroup$ – Łukasz Grad Mar 9 '17 at 23:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.