When testing the difference between 2 proportions why do we use a z-test rather than a t-test?

Further, is there a simple way to conduct an omnibus test for significant differences between more than 2 proportions (in the form of percentages). Is there an equivalent to a one-way ANOVA for this? I imagine that you could use logistic regression (assuming you have the original data for the proportions in the form of 0s and 1s) Are there other options?

  • $\begingroup$ If $\hat p = s/n$ estimates $p.$ then $np$ estimates population mean and $np(1-p)$ estimates population variance. // In a t statistic the mean and variance are estimated by stochastically independent quantities. $\endgroup$ – BruceET May 1 at 19:58

Short version: You don't use a t-test because the obvious statistic doesn't have a t-distribution. It does (approximately) have a z-distribution.

Longer version:

In the usual t-tests, the t-statistics are all of the form: $\frac{d}{s}$, where $s$ is an estimated standard error of $d$. The t-distribution arises from the following:

1) $d$ is normally distributed (with mean 0, since we're talking about distribution under $H_0$)

2) $k.s^2$ is $\chi^2$, for some $k$ (I don't want to belabor the details of what $k$ will be, since I'm covering many different forms of t-test here)

3) $d$ and $s$ are independent

Those are a pretty strict set of circumstances. You only get all three to hold when you have normal data.

If, instead, the estimate, $s$ is replaced by the actual value of the standard error of $d$ ($\sigma_d$), that form of statistic would have a $z-$distribution.

When sample sizes are sufficiently large, a statistic like $d$ (which is often a shifted mean or a difference of means) is very often asymptotically normally distributed*, due to the central limit theorem.

* more precisely, a standardized version of $d$, $d/\sigma_d$ will be asymptotically standard normal

Many people think that this immediately justifies using a t-test, but as you see from the above list, we only satisfied the first of the three conditions under which the t-test was derived.

On the other hand, there's another theorem, called Slutsky's theorem that helps us out. As long as the denominator converges in probability to that unknown standard error, $\sigma_d$ (a fairly weak condition), then $d/s$ should converge to a standard normal distribution.

The usual one and two-sample proportions tests are of this form, and thus we have some justification for treating them as asymptotically normal, but we have no justification for treating them as $t$-distributed.

In practice, as long as $np$ and $n(1-p)$ are not too small**, the asymptotic normality of the one and two-sample proportions tests comes in very rapidly (that is, often surprisingly small $n$ is enough for both theorems to 'kick in' as it were and the asymptotic behavior to be a good approximation to small sample behavior).

** though there are other ways to characterize "large enough" than that, conditions of that form seem to be the most common.

While we don't seem to have a good argument (at least not that I have seen) that would establish that the t should be expected to be better than the z as an approximation to the discrete distribution of the test statistic at any particular sample size, nevertheless in practice the approximation obtained by using a t-test on 0-1 data seems to be quite good, as long as the usual conditions under which the z should be a reasonable approximation hold.

is there a simple way to conduct an omnibus test for significant differences between more than 2 proportions (in the form of percentages)

Sure. You can put it into the form of a chi-square test.

(Indeed, akin to ANOVA you can even construct contrasts and multiple comparisons and such.)

It's not clear from your question, however, whether your generalization will have two samples with several categories, or multiple samples with two categories (or even both at once, I guess). In either case, you can get a chi-square. If you are more specific I should be able to give more specific details.

| cite | improve this answer | |
  • 2
    $\begingroup$ +1, this is a good answer, much more thorough than mine. I'm cautious about the chi-squared test, though. The thing about the chi-squared test is that it doesn't assume one of the variables is an explanatory variable & one is a response variable, so if that's true, you have less power than say a logistic regression w/ all categorical predictors. I describe this in the other answer I link to in my answer here. $\endgroup$ – gung - Reinstate Monica Mar 21 '14 at 23:56
  • 1
    $\begingroup$ @gung as I read it, the question that is posed is equality of probabilities across categories ("an omnibus test for significant differences between more than 2 proportions"). The chi-square does this. It's actually possible to formulate a wide variety of comparisons-of-proportions problems, with multiple factors (/explanatory variables) as ANOVA-like chi-squares (one-way, two-way and so on), with partitions of sums of squares replaced with partitions of chi-squares. You may be surprised at quite how far the analogy goes. $\endgroup$ – Glen_b Mar 22 '14 at 0:13
  • $\begingroup$ fantastic answer - this should be made the canonical answer to which point all the other ones related to t-test of proportion. $\endgroup$ – Xavier Bourret Sicotte Nov 13 '18 at 6:09
  • 1
    $\begingroup$ +1 @gung-ReinstateMonica glen_b when you write "an omnibus test for significant differences between more than 2 proportions" it is probably important to add the explicit qualifier "in unblocked data": the omnibus test for proportions across blocked data is Cochran's Q test (though you can use McNemar's test when you have only two blocks... i.e. the data are 'paired'). Just as there is vanilla ANOVA and repeated measures ANOVA for approximately normal data with = vars, there is $\chi^{2}$ and Cohran's Q for binomial proportions. $\endgroup$ – Alexis Nov 7 '19 at 0:07
  • $\begingroup$ Ann points (in a deleted answer) to: "The Appropriateness of Some Common Procedures for Testing the Equality of Two Independent Binomial Populations by Ralph B. D'agostino,Warren Chase &Albert Belanger. The authors write "In this article we demonstrate that, even for small samples, the uncorrected chi-squared test (i.e., the Pearson chi-squared test) and the two-independent-sample t test are robust in that their actual significance levels are usually close to or smaller than the nominal levels. We encourage the use of these latter two tests." $\endgroup$ – Glen_b May 10 at 0:13

The reason you can use a $z$-test with proportion data is because the standard deviation of a proportion is a function of the proportion itself. Thus, once you have estimated the proportion in your sample, you don't have an extra source of uncertainty that you have to take into account. As a result, you can use the normal distribution instead of the $t$ distribution as your sampling distribution. To learn more about this, see my answer here: The $z$-test vs the $\chi^2$-test for comparing the odds of catching a cold in 2 groups.

If you have more than 2 groups, you can use logistic regression, as you note. You do have to know the $n_j$s in each group however. If you just had a set of observed proportions, but didn't know how many trials had been observed to generate those proportions, you cannot run a proper test of whether the proportions differed.

| cite | improve this answer | |
  • $\begingroup$ I thought that the t-test was for normally distributed data, with unknown standard deviation, and gives you an exact test (or confidence interval), whereas the z-test is for approximately normal distributed data, and gives an approximate test (or confidence interval) -- or is this saying the same thing as your answer but in a different way? $\endgroup$ – TooTone Mar 21 '14 at 18:49
  • $\begingroup$ You can see more in my linked answer, @TooTone. Both tests compare the calculated test statistic to a sampling distribution assumed for that test statistic. If your test statistic was calculated from a sample of data from a normal population, but where both the mean & SD needed to be calculated from the sample, both sources of uncertainty need to be taken into account, & it turns out that the sampling distribution will be $t$. If you don't have to worry about uncertainty in the SD as well, b/c it's known a-priori or your sample is infinite, then the sampling distribution will be normal. $\endgroup$ – gung - Reinstate Monica Mar 21 '14 at 21:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.