Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
When testing the difference between 2 proportions why do we use a z-test rather than a t-test?
Further, is there a simple way to conduct an omnibus test for significant differences between more than 2 proportions (in the form of percentages). Is there an equivalent to a one-way ANOVA for this? I imagine that you could use logistic regression (assuming you have the original data for the proportions in the form of 0s and 1s) Are there other options?
Short version: You don't use a t-test because the obvious statistic doesn't have a t-distribution. It does (approximately) have a z-distribution.
Longer version:
In the usual t-tests, the t-statistics are all of the form: $\frac{d}{s}$, where $s$ is an estimated standard error of $d$. The t-distribution arises from the following:
1) $d$ is normally distributed (with mean 0, since we're talking about distribution under $H_0$)
2) $k.s^2$ is $\chi^2$, for some $k$ (I don't want to belabor the details of what $k$ will be, since I'm covering many different forms of t-test here)
3) $d$ and $s$ are independent
Those are a pretty strict set of circumstances. You only get all three to hold when you have normal data.
If, instead, the estimate, $s$ is replaced by the actual value of the standard error of $d$ ($\sigma_d$), that form of statistic would have a $z-$distribution.
When sample sizes are sufficiently large, a statistic like $d$ (which is often a shifted mean or a difference of means) is very often asymptotically normally distributed*, due to the central limit theorem.
* more precisely, a standardized version of $d$, $d/\sigma_d$ will be asymptotically standard normal
Many people think that this immediately justifies using a t-test, but as you see from the above list, we only satisfied the first of the three conditions under which the t-test was derived.
On the other hand, there's another theorem, called Slutsky's theorem that helps us out. As long as the denominator converges in probability to that unknown standard error, $\sigma_d$ (a fairly weak condition), then $d/s$ should converge to a standard normal distribution.
The usual one and two-sample proportions tests are of this form, and thus we have some justification for treating them as asymptotically normal, but we have no justification for treating them as $t$-distributed.
In practice, as long as $np$ and $n(1-p)$ are not too small**, the asymptotic normality of the one and two-sample proportions tests comes in very rapidly (that is, often surprisingly small $n$ is enough for both theorems to 'kick in' as it were and the asymptotic behavior to be a good approximation to small sample behavior).
** though there are other ways to characterize "large enough" than that, conditions of that form seem to be the most common.
While we don't seem to have a good argument (at least not that I have seen) that would establish that the t should be expected to be better than the z as an approximation to the discrete distribution of the test statistic at any particular sample size, nevertheless in practice the approximation obtained by using a t-test on 0-1 data seems to be quite good, as long as the usual conditions under which the z should be a reasonable approximation hold.
is there a simple way to conduct an omnibus test for significant differences between more than 2 proportions (in the form of percentages)
Sure. You can put it into the form of a chi-square test.
(Indeed, akin to ANOVA you can even construct contrasts and multiple comparisons and such.)
It's not clear from your question, however, whether your generalization will have two samples with several categories, or multiple samples with two categories (or even both at once, I guess). In either case, you can get a chi-square. If you are more specific I should be able to give more specific details.
The reason you can use a $z$-test with proportion data is because the standard deviation of a proportion is a function of the proportion itself. Thus, once you have estimated the proportion in your sample, you don't have an extra source of uncertainty that you have to take into account. As a result, you can use the normal distribution instead of the $t$ distribution as your sampling distribution. To learn more about this, see my answer here: The $z$-test vs the $\chi^2$-test for comparing the odds of catching a cold in 2 groups.
If you have more than 2 groups, you can use logistic regression, as you note. You do have to know the $n_j$s in each group however. If you just had a set of observed proportions, but didn't know how many trials had been observed to generate those proportions, you cannot run a proper test of whether the proportions differed.
