I'm looking for a suitable test for the difference in two proportions when dealing with finite populations. The standard z-test seems to only apply to infinite populations, so you lose some power.

It is of course possible to just do two confidence intervals with the finite population correction and check that they don't overlap, but this also tends to lose power compared to a direct comparison I think?

Would greatly appreciate if someone could point me in the right direction!

  • $\begingroup$ In addition to the accepted answer, beware that doing two confidence intervals and checking if they overlap is in general not equivalent to any test about the difference. $\endgroup$ – Pere Jan 18 '17 at 9:10
  • $\begingroup$ Thanks - I do know that, but I think it's reasonable to assume when confidence intervals don't overlap that there's a low probability of both having the same population mean? In general it's weaker than doing a standard test (i.e. no overlap generally means signficance, but overlap doesn't always imply no significance), but with the fpc it can be close. $\endgroup$ – Cian Murphy Jan 19 '17 at 11:10

It's quite complicated to test for two proportions in a complex survey data, that's because there is no closed formula for the standard errors of the proportion estimators. Generally, you'd need to estimate the standard errors with bootstrap, balanced replicated method or jackknife.

I recommend Applied Survey Data Analysis. The book has a section about testing for sample means with complex t-test. Testing for proportion is a simple extension because proportion to related to mean by the sample size.

Basically, you will need:

  1. Use FPC and inverse probability of selection to estimate weights for each response
  2. The unbiased estimate for the proportion is given by the normal formula with your weighted responses
  3. The variance for your estimators require the covariance of your estimators. I took a photo for section 5.6 in the book for you:

$$ \newcommand{var}{\text{var}} \newcommand{cov}{\text{cov}} \var\Big( \sum_{j=1}^{J} a_j \hat\theta_j \Big) = \sum_{j=1}^{J} a_j^2 \var({\hat\theta_j}) + 2 \times \sum_{j=1}^{J-1}\sum_{k>j}^{J} a_j a_k \cov(\hat\theta_j, \hat\theta_k) $$

The formula gives the general formula for linear combination of your estimators. Testing for proportion fit into the framework.

As you can see, you need the covariance for your estimators. You should use a numerical method such as bootstrap to estimate it.

  1. Once you have the unbiased difference and it's variance (i.e. standard error). You form your test statistic simply by dividing your difference with it's standard error. Compute p-value like how you would do for a simple random sampling survey.

The survyemean function in SAS can do everything for you.


Note that you can choose what numeric method to use.

Please take a look at the book for the theories.

| cite | improve this answer | |
  • $\begingroup$ Very helpful, thanks. Didn't realise it would be so complicated! $\endgroup$ – Cian Murphy Jan 17 '17 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.