Testing equality of two binomial proportions proportion (one near 100 %)

I am testing equality of two binomial proportions 87/88 and 48/60. I use this online calculator and it noted that the Standard Normal approximation is not valid in this case. It seem quite strange to me. Is it some problem in testing equality of two binomial proportions if one of the proportions is very near to 100 %?

-

If you trust wikipedia for rules of thumb for the validity of the normal approximation of the binomial distribution:

• One rule is that both $np$ and $n(1−p)$ must be greater than $5$. However, the specific number varies from source to source, and depends on how good an approximation one wants; some sources give $10$ which gives virtually the same results as the following rule for large ''n'' until ''n'' is very large (ex: ''x=11, n=7752'').

• A second rule is that for $n > 5$ the normal approximation is adequate if

$$\left|(1/\sqrt{n})(\sqrt{(1-p)/p}-\sqrt{p/(1-p)})\right|<0.3$$

• Another commonly used rule holds that the normal approximation is appropriate only if everything within 3 standard deviations of its mean is within the range of possible values that is if

$$\mu \pm 3 \sigma = np \pm 3 \sqrt{np(1-p)} \in [0,n]. \,$$

All of those fail when $p$ is close to either $0$ or $1$. The intuitive idea is that then the distribution is:

1. very skewed, and
2. the normal approximation will be too significant outside the actual bounds of the binomial distribution, $[0, n]$.
-
Thank you very much for your answer. Now I understand that my distribution is very skew and normal approximation is not valid. However in my opinion proportions 87/88 and 48/60 look very different. Can I use another test to prove difference between proportions? – mcihak Jan 18 '13 at 0:17

+1 for @Jaime. But as it happens in this case your null hypothesis is that both proportions equal a pooled figure of (87+48)/(88+60) = 0.91. With your sample size this is in the acceptable area for approximations such as this z test or the equivalent chi square test. See that the values in the "expected" (meaning expected under the null hypothesis of equal proportions) matrix below are all more than 5, usually accepted as an ok rule of thumb.

I would advocate as a simple solution a Chi square test with continuity correction - which agrees with you (low p value) that it is unlikely a common underlying proportion would produce these two observed sets of data.

> p <- (87+48)/(88+60)
> p
[1] 0.9121622
> obs <- matrix(c(87,1,48,12), nrow=2)
> obs
[,1] [,2]
[1,]   87   48
[2,]    1   12
> expected <- rbind(p * margin.table(obs,2),(1-p) * margin.table(obs,2))
> expected
[,1]     [,2]
[1,] 80.27027 54.72973
[2,]  7.72973  5.27027
> chisq.test(obs)

Pearson's Chi-squared test with Yates' continuity correction

data:  obs
X-squared = 13.5773, df = 1, p-value = 0.0002289

-
+1 I suspected that would turn out to be the null hypothesis, but was not sure. But once you know what the population proportion should be, methinks it shouldn't be too hard to come up with exact probabilities from the binomial distribution itself, right? – Jaime Jan 18 '13 at 2:29
You can but it can be slightly awkward to turn this into a hypothesis test. The probability of 87 or higher out of 88 with your combined observed proportion is 0.0029; and of 48 or less is 0.0054; but there are philosophical problems with combining these into a single p value for how unlikely these data are under the null. Could be done with care, thinking carefully about 2 tailed v 1 tailed tests and the like, and what it means to have snooped on your data. For the record, in R use something like sum(dbinom(87:88, 88, 0.9121622)) for the calculation of the binomial probability. – Peter Ellis Jan 18 '13 at 3:00