0
$\begingroup$

I have data collected over two years where:

  • Year 1 = 34 successes in the representative sample of 49 taken from the finite population of 122, and

  • Year 2 = 45 successes in the representative sample of 68 taken from the finite population of 175.

What test do I use to see if there is any significant difference between the two?

$\endgroup$
4
  • $\begingroup$ You'll want to use a two sample test for difference in proportions: onlinecourses.science.psu.edu/stat414/node/268 $\endgroup$
    – mai
    Commented Mar 12, 2018 at 2:56
  • $\begingroup$ Thank mchen. This test doesn't take into account the statistically representative sample size of the total population from within which the successes were achieved - can you test the successes on the total population without needing to consider the sample size within which they were achieved? $\endgroup$
    – user198406
    Commented Mar 12, 2018 at 3:24
  • 2
    $\begingroup$ Would simulation be a reasonable answer? 34 successes in 49 gives you a distribution of credible values for the rate $\theta_1$ in the first year. so credible values for the successes in the population in year 1 are 49 plus binomial samples taken from 122 - 49 = 73 individuals. This can easily be simulated. The same can be done for Year 2 and then you could compare. That is, if simulation is suitable for the problem behind the scenes. $\endgroup$
    – Bernhard
    Commented Mar 12, 2018 at 13:37
  • $\begingroup$ @Bernhard I am with you on this. I think that a parametric bootstrap approach is warranted. A random sample is drawn as described. In group 1, 34/15 of the 122 are definitive successes/failures and the remaining 73 are Bernoulli according to $p=34/49$. In group 2 similarly. I note you call them credible intervals, and as an ardent frequentist, am struggling to justify a frequentist approach. I might agree it's a fundamentally Bayesian hypothesis and approach. $\endgroup$
    – AdamO
    Commented Apr 13, 2018 at 14:32

1 Answer 1

1
$\begingroup$

This is a difficult problem. Please review this article:

Krishnamoorthy, K. and Thompson, J. 2002. "Hypothesis Testing about Proportions in Two Finite Populations". The American Statistician 56(3): 215-223

This is a difficult issue because the sampling distribution of estimated proportions is not known. For many years, it was approximated by a normal distribution, mostly because it was workable. Since 1990, it has been recognized that the usual CI method, m +/- 1.96 std.err(m) is tragically wrong for proportions and there is a lot of hand wringing about what to do. This article amazed me:

Brown, Lawrence D, Cai, Tony T, and DasGupta, Anirban. 2001. Interval Estimation for a Binomial Proportion. Statistical Science 16(2):101-133

I'm not a specialist and, like you, I have just wanted to know "the answer", but, sadly, no definitive answer is forthcoming. There are more than 20 proposed methods of making confidence intervals and each one of those ideas can be re-wrapped to make a hypothesis test.

From all of this, we are absolutely sure that the normal-based approximation is invalid and we have suggestions for methods that are demonstrably better. However, no method is demonstrably the best and thus we can't tell you precisely what to do.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.