Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
What is the best way to test for difference of proportions when of them is 0% (or 100%) ?
edit: I have two raters, who give a score of either 1 or 0. One of them gave a 0 in every single case. Now I want to know if the proportions of 1's is significantly different between the two.
Since you are referring to the sample proportion and not the population proportion. The case of sample proportion equal 0 does not create problems unless you try to use normal or chi-square approximations. You can apply Fisher's exact test for differences in proportions or apply the Kappa statistic to test interrater agreement
If, as I suspect from the context, the samples aren't huge, then this issue is not so straightforward. This article summarizes some of the issues involved, and includes many references. It ends by recommending the score method of Miettinen and Nurminen, but bear in mind that one of the authors of the paper is Nurminen.
Another very useful reference is Brown, Cai & DasGupta (2001). They write (in the abstract):
Based on this analysis, we recommend the Wilson interval or the equal-tailed Jeffreys prior interval for small n and the interval suggested in Agresti and Coull for larger n. We also provide an additional frequentist justification for use of the Jeffreys interval.
and note that the coverage of the standard methods is often bad, even when N is large, and that (very unusually) there is not a monotonic relationship between the coverage and the sample size (that is, larger samples can be "worse" than smaller ones).
For looking at a single binomial proportion, a good article is Agresti, 1998. This may be useful background for the other two articles, which are more directly on point for your question.
