First, this seems like a very common question, but I do promise that I've searched for a long time through other similar posts and haven't quite found a solution that seems to provide an answer. (This and this come very close, but I don't think either quite answers it!).
I'll be very thorough in explaining my confusion, by comparing it first to the one-sample proportion case. We could test whether the true proportion $p$ equals some value $c$.
$H_0: p = c.$
$H_A: p \neq c.$
For $np$ and $n(1-p)$ sufficiently large, the binomial approximation to the normal gives us $\hat{p} \sim N(p, p(1-p)/n)$. Thus, when we compute the test statistic, $Z := \frac{\hat{p} - c}{\sqrt{c(1-c)/n}}.$ Under the null hypothesis this is distributed as standard normal. Crucially, we are not estimating the standard error–it is determined by the null hypothesis!
Now, we instead consider the two-sample case, where we want to run a hypothesis test on the difference in proportions.
$H_0: p_1 - p_2 = 0.$
$H_A: p_1 - p_2 \neq 0.$
The same binomial approximation gives us $\hat{p}_i \sim N(p_i, p_i(1-p_i)/n_i)$, $i=1,2$. Then, if $\tilde{p}$ is the pooled proportion ($\hat{p} = (x_1 + x_2)/(n_1+n_2)$), I know that our test statistic is given by $Z := \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\tilde{p}(1-\tilde{p})(1/n_1 + 1/n_2)}}.$
This is the crucial part I do not follow. In the one-sample case, we did not estimate the standard error–it was determined by the null. Hence, citing the standard normal makes sense. In the two-sample case, we do have to estimate the standard error, using our pooled proportion! So, basically, shouldn't we have to make some sort of adjustment to reflect this? (I.e., like what we do with a t-test).
So, how can I explain this? Why can we still cite the standard normal, even though we are using a data-based estimate for its standard error?
While the question is analogous to that of a t-test, I understand why we can't just use a t-test (we don't have the other assumptions met). My best attempt at an answer so far is simply:
“Yes, we are estimating the standard error, but that approximation is just swallowed up by our original normal approximation to the binomial, because that only works at high degrees of freedom anyways"?
Is there a better explanation that that? It feels intuitive that some adjustment would be necessary, but we don't make one.
For a brief explanation of why the two links I cited felt like they didn't quite cover this...
@glen_b's answer is very good, and clearly explains why the theoretical assumptions required for a t-test wouldn't hold here. However, I don't see it explain this exact issue. It mentions both one and two sample proportion tests, and my confusion is that they don't feel like they are the same on this front. But this helps inform my best guess above, which is that for sample sizes large enough for our normality approximation, estimating the standard error is basically irrelevant.
Similarly, @whuber's very clearly shows (with plots) how the student-t distribution doesn't approximate our test statistic any better (for moderate sample size) than the simple standard normal.
So, I understand why the student-t distribution isn't a better choice here. But my lingering confusion is simply: even if the student-t isn't the right fix, what's the best way to describe why we don't provide any sort of adjustment here? I see the one sample and two sample case described equivalently–"it's normal because the variance is determined by the mean". But the cases seem quite different–in one, we estimate the standard error, in the other, we don't. Is the answer simply "once we are already approximating the binomial proportion difference with a normal, the estimation of the standard error is trivial relative to that approximation, so we can ignore it"?
