3
$\begingroup$

When are you supposed to use the pooled formula vs the unpooled formula? My understanding is that you use the pooled formula when testing a hypothesis where the null hypothesis is $H_0 = 0$ and the alternative hypothesis is $H_0 \ne 0$. My confusion arises when doing hypothesis testing for "greater than" or "less than" scenarios. Do you pool it to create a ${\hat{p}}$ or do you use ${p_1}$ and ${p_2}$ seen in the unpooled formula below?

Which formula would I use if I were testing:

$H_0 = 0$

$H_a \gt 0$

Or:

$H_0 = 0$

$H_a \lt 0$

Or other scenarios like:

$H_0 \ge n$

$H_a \lt n$

Or:

$H_0 \le n$

$H_a \gt n$

Pooled Formula:

$z = \dfrac{({p_1} - {p_2}) - d_0}{\sqrt{\hat{p} (1-\hat{p}) \left(\dfrac{1}{n_1} + \dfrac{1}{n_2} \right)}}$

Where $\hat{p} = \dfrac{x_1 + x_2}{n_1 + n_2}$

Unpooled Formula:

$z = \dfrac{({p_1} - {p_2}) - d_0}{\sqrt{\dfrac{{p_1}(1-{p_1})}{n_1} + \dfrac{{p_2}(1-{p_2})}{n_2}}}$

Where ${p_1}$ and ${p_2}$ are simply the proportions in decimal format.


I see so many confusing posts on this subject:

[1] Video maker tests null=0, alternative>0 and uses pooled

[2] This StackExchange post suggests there is not a consensus among stats practicioners

[3] This StackExchange post suggests you use unpooled when you "believe" the population proportions are different - I have a problem with the word "believe" as I'm looking for a concrete definitive rule to guide my decision framework.

[4] Makes the case that pooled estimates give us the best estimate for variability - but doesn't have examples of "greater than" or "less than" scenarios which makes it hard for me to understand

[5] StackExchange post provides good examples of the difference in calculation but also states 'the final statistical decision does not quite exist on this'

[6] YouTube video showing a two proportion left-tail z-test - Video maker uses pooled formula when $H_a \lt 0$ .

[7] YouTube video shows using pooled formula in one-sided test

[8] YouTube video computes $\hat{p}$ but uses in unpooled formula

[9] Blog writer uses unpooled formula to calculate the standard error for a one-tailed test

... I'll stop there.


Edit (1 year later):

Here's another example. There's a popular online tool that is often referenced as the "go-to" source for A/B testing between two-sample proportions.

Their tool doesn't pool but all over the page they are referencing the DIFFERENCE... which to me, implies a null hypothesis of no difference in which case they should be using a pooled estimate.

Example:

s1 = 5
n1 = 1023

s2 = 4
n2 = 344

p1 = s1 / n1
p2 = s2 / n2

# abtestguide.com calculation of z-score
z_unpooled = (p1 - p2) / (((p1*(1-p1))/n1) + ((p2*(1-p2))/n2))**0.5

# pooled calculation of z-score
p = (s1 + s2) / (n1 + n2)
z_pooled = (p1 - p2) / (p*(1-p)*((1/n1)+(1/n2)))**0.5

print('Unpooled: {:.6f}, Pooled: {:.6f}'.format(z_unpooled, z_pooled))
# Unpooled: -1.091082, Pooled: -1.337244

Can anyone help clarify when to use pooled vs unpooled? I've read to use pooled unless you have a good reason not to. I've also read that you use unpooled when testing for "better" or "worse" criteria. It's so unclear to me.

$\endgroup$
  • $\begingroup$ Usually we write something like $H_o: \theta = \theta_o$ versus $H_1: \theta \ne \theta_o$ for some well-defined parameter $\theta$. Writing $H_o = 0$ does not look right. $\endgroup$ – Michael M Jul 27 '16 at 17:52
  • $\begingroup$ Feel free to edit. theta doesn't mean anything to me and 0 makes more sense since we're saying there is no difference between two samples so the difference equals zero. $\endgroup$ – Jarad Jul 27 '16 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.