# Implicit assumption in z-test of difference in proportions?

In reading through the literature on z-tests for difference in proportions, I often see it stated that the null hypothesis only tests whether p1 = p2, not whether p1 = p2 = any specific value for the pooled proportion. But in fact when we use the overall/pooled proportion from our sample to estimate the population proportion to calculate our test statistic, are we not actually making a concrete assumption about a specific value that p1 and p2 are both equal to?

As an illustration, suppose a researcher is interested in evaluating whether men or women in a large city are more likely to own a pet. The researcher performs separate, independent random samples of 20 men and 20 women and records whether they own a pet or not. Now lets say the results are that 8 of the 20 men own pets, and 12 of the 20 women own pets. The overall (pooled) proportion of pet owners from this exercise is 20/40 or 0.5. By using 0.5 to calculate our test statistic, aren't we really testing the probability that p1 = p2 = 0.5? There is nothing in the standard calculation of the test statistic which says anything about the probability that p1 = p2 = 0.2, or 0.8, or any other value other than 0.5.

It seems to me that if you wanted to really test the probability p1 = p2 = any specific value, you would need to run separate tests for specific different possible p1=p2 values, and then weight the outcomes by the likelihood of these different possible prior probabilities. But that begs the question of what the distribution of prior probabilities looks like....

But really rather than getting into a long discussion on a much more complex analysis involving estimating alternative prior probability distributions, I really am looking for confirmation that the more common approach of using a z-test for difference in proportions is actually testing the validity of a null hypothesis that p1 = p2 = the specific value as determined by the pooled sample proportion - despite what I have commonly encountered in textbooks and online explanations which claim that no specific value for p1 = p2 is being tested.

• An astute question. See also On Fisher's exact test: What test would have been appropriate if the lady hadn't known the number of milk-first cups? for some discussion of exact tests. Nuisance parameters can be a nuisance. May 26, 2016 at 19:13
• Thanks for your comment! Since first learning about the FET, I have always felt it to be a very exact answer in search of proper question. I have tried to calculate a p-values more directly by simply looking at all the possible discrete outcomes for the product of two binomials, and summing all outcomes which have an equal or lower probability of occurring compared to the specific outcome observed in the 2 independent samples. After all, isn't the purpose of hypothesis testing to see just how rare the outcome we have observed must be if the null is true? May 26, 2016 at 20:00
• Comments are not for extended discussion; this conversation has been moved to chat. May 27, 2016 at 9:54

The textbooks are right and you test $p_1=p_2$ vs. them not being equal. The standard test statistic is a quantity., for which the distribution is asymptotically known under the null hypothesis, which is the basis for using it. It so happens to involve the observed proportions (unsurprisingly, really), but the value of $p_1$ is just a nuissance parameter.
• Using a different number would give a different test statistics, but under repeated sampling using this formula gives a level $\alpha$ test. Or frame it as a logistic regression, where we assume $Y_i \sim \text{Bin}(n_i, p)$ and $\text{logit}(p) = \text{logit}(p_1) + 1{i=2} \times \beta$. If you do a hypothesis test for the log-odds ratio $\beta$, you are not doing a hypothesis test for $\text{logit}(p_1)$ and you could commit a separate type I error for it. Another thing is that $p_1 = p_2 = \text{sample proportion}$ is not an up-front hypothesis (depends on the sample proportion!). May 26, 2016 at 19:02