# How to compare groups with different sizes?

I need to calculate the winner of a PTA membership contest for our local elementary school. There are 15 classes of different sizes (size based on number of students in the class). We need to see which class has won the membership contest. I have data like the following:

Grade 1: 5 members out of 13 students Grade 2: 3 members out of 22 students Grade 3: 12 members out of 30 students Grade 4: 1 out of 2 students

I was originally just calculating percentages and the class with the highest membership percentage would win the contest but, this doesn't seem right. Grade 4 only has 2 students and just 1 member means that they would win the contest while Grade 3 has a total of 12 new members and they come up short based on the percentage calculation. This doesn't seem fair because the 3rd grade teacher made a huge effort to get the parents of the kids in her class to become members of the PTA. Is there a better way to compare these groups?

• I don't think this is really a statistics question. I think it's about "what else can I do so that I get the result I want instead of the one I don't like?" If there are rules for this competition, you have to follow them. If there are not, then now it's obvious why there should have been. I can't imagine a school with 30 3rd-graders and only 2 4th-graders, but I guess that's beside the point. – rvl Dec 16 '14 at 17:34
• I am running the competition and I want it to be fair. I am not trying to tweak the results. And yes...there are some classes with under 5 students. These are special needs classrooms. – Ivan Dec 17 '14 at 4:37

One possibility (if you can do it) is to jointly award a prize to the greatest number and the greatest proportion. People will likely find that easier to accept that what I am about to talk about:

There are some ways you can 'adjust' for the fact that very small proportions can be 'noisy' (which noisiness causes a problem if there are numerous small groups -- the highest of them will nearly always be higher than any of the big groups).

This is to compare some proportion you can be reasonably "confident"* has been exceeded (the lower limit of some interval for the proportion).

* this is an abuse of terminology, since we're dealing with frequentist intervals; maybe I should have presented an approximate Bayesian one.

For example, an approximate 68% two-sided interval (about 5/6 chance the true proportion will exceed it) for a proportion could be obtained from looking at an approximation to an Agresti-Coull type approach. At that point it's basically "add 1 to both the successes and failures, and take an asymptotic one-sided interval (using a normal approximation).

That is, if $X$ is the number of "successes" and $n$ is the number of students in a class, then $X^+=X+1$ and $n^+=n+2$, and $p^+=X^+/n^+$, then you compare $p^\text{lower}=p^+-\sqrt{p^+(1-p^+)/n^+}$ and whichever is the highest "wins".

The big problem with this approach is its not at all transparent. If you have to explain it to anyone, it looks like a complete fudge.

However, maybe you could explain taking the upper bound on the second term like they do when calculating margin of error in polls: in that way, perhaps you can kind of justify something simpler like $p-.5/\sqrt{n}$ (I wouldn't suggest the more usual 1.96 calculation - not only is it notionally "too much" of a penalty to my mind, some groups would go negative!) as a "margin of error adjusted proportion" on the raw proportions. It no longer has any real accuracy as a true confidence interval interpretation but maybe that doesn't matter. (For that matter you could call the more complicated calculation a margin-of-error adjusted proportion too.)

[Alternatively you could take one of those results and scale the smallest group back up to 50% and add the same proportion to everyone else, and call it a "count-adjusted proportion" or something.]

Frankly, to most people anything like this would still look like a fudge. I'd avoid it if there's any way to do so, but it is one way of adjusting in some sense for the tendency of very small groups to win such things. It's generally best to telegraph such shennanigans 'up front' wherever possible..