When are you supposed to use the pooled formula vs the unpooled formula? My understanding is that you use the pooled formula when testing a hypothesis where the null hypothesis is $H_0 = 0$ and the alternative hypothesis is $H_0 \ne 0$. My confusion arises when doing hypothesis testing for "greater than" or "less than" scenarios. Do you pool it to create a ${\hat{p}}$ or do you use ${p_1}$ and ${p_2}$ seen in the unpooled formula below?
Which formula would I use if I were testing:
$H_0 = 0$
$H_a \gt 0$
Or:
$H_0 = 0$
$H_a \lt 0$
Or other scenarios like:
$H_0 \ge n$
$H_a \lt n$
Or:
$H_0 \le n$
$H_a \gt n$
Pooled Formula:
$z = \dfrac{({p_1} - {p_2}) - d_0}{\sqrt{\hat{p} (1-\hat{p}) \left(\dfrac{1}{n_1} + \dfrac{1}{n_2} \right)}}$
Where $\hat{p} = \dfrac{x_1 + x_2}{n_1 + n_2}$
Unpooled Formula:
$z = \dfrac{({p_1} - {p_2}) - d_0}{\sqrt{\dfrac{{p_1}(1-{p_1})}{n_1} + \dfrac{{p_2}(1-{p_2})}{n_2}}}$
Where ${p_1}$ and ${p_2}$ are simply the proportions in decimal format.
I see so many confusing posts on this subject:
[1] Video maker tests null=0, alternative>0 and uses pooled
[2] This StackExchange post suggests there is not a consensus among stats practicioners
[3] This StackExchange post suggests you use unpooled when you "believe" the population proportions are different - I have a problem with the word "believe" as I'm looking for a concrete definitive rule to guide my decision framework.
[4] Makes the case that pooled estimates give us the best estimate for variability - but doesn't have examples of "greater than" or "less than" scenarios which makes it hard for me to understand
[6] YouTube video showing a two proportion left-tail z-test - Video maker uses pooled formula when $H_a \lt 0$ .
[7] YouTube video shows using pooled formula in one-sided test
[8] YouTube video computes $\hat{p}$ but uses in unpooled formula
[9] Blog writer uses unpooled formula to calculate the standard error for a one-tailed test
... I'll stop there.
Edit (1 year later):
Here's another example. There's a popular online tool that is often referenced as the "go-to" source for A/B testing between two-sample proportions.
Their tool doesn't pool but all over the page they are referencing the DIFFERENCE... which to me, implies a null hypothesis of no difference in which case they should be using a pooled estimate.
Example:
s1 = 5
n1 = 1023
s2 = 4
n2 = 344
p1 = s1 / n1
p2 = s2 / n2
# abtestguide.com calculation of z-score
z_unpooled = (p1 - p2) / (((p1*(1-p1))/n1) + ((p2*(1-p2))/n2))**0.5
# pooled calculation of z-score
p = (s1 + s2) / (n1 + n2)
z_pooled = (p1 - p2) / (p*(1-p)*((1/n1)+(1/n2)))**0.5
print('Unpooled: {:.6f}, Pooled: {:.6f}'.format(z_unpooled, z_pooled))
# Unpooled: -1.091082, Pooled: -1.337244
Can anyone help clarify when to use pooled vs unpooled? I've read to use pooled unless you have a good reason not to. I've also read that you use unpooled when testing for "better" or "worse" criteria. It's so unclear to me.
