# How to compare two reviews, when one has more participants than the other

Hi I only did some basic stats at school but now I need to rank some data for a project, and I've run into this kind of scenario

Game A: 1000 Reviews, 900 Positive and 100 Negative, so 90% positive.

Game B: 20 Reviews, 20 Positive and 0 Negative, so 100% positive.

It clearly isn't fair to say Game B is better than Game A just because it have a higher percentage of positive reviews, but I still need a way to conclude their positive percentage. How should I do so?

EDIT: In my case, it's more than just two data, it's about hundred or thousands of data, some value are just extremely large and some are small.

• For each game , let $p$ be the probability of positive review (one different $p$ for each different game). Tou could present confidence intervals for each $p$. If you have many games and want a ranking, there are statistical methods for that too, but maybe complicated with only "basic stats". Then you would need to formulate more precisely your goal. Can you post a link to your data? Aug 26, 2015 at 17:15
• One approach that has been used is to calculate a "lower bound" on the positive proportion based on a binomial proportion confidence interval. Evan Miller discussed the Wilson interval, but any of the others could be used. This sort of approach has been widely used for comparing various kinds of ratings (e.g. I gather it's used by reddit in ranking comments). ... ctd Aug 26, 2015 at 22:54
• ctd... This is the subject of a number of posts on our site, but you should probably start with this one since the discussion there (at least briefly) mentions problems and alternatives. [Edit -- It looks like reddit's use of it had - at the time this was written - some implementation issues.] Aug 26, 2015 at 23:12

Because the standard Binomial confidence interval doesn't work when $$p=0$$ or $$p=1$$ you can try using the exact binomial confidence interval to calculate a confidence score for the 1st and second games and then compare the scores.

from scipy.stats import beta

# calculate lower bound for game A
lower_a = 1-beta.ppf(.975, 101, 900)
# calculate upper bound for game A
upper_a = 1-beta.ppf(0.025, 100, 901)

# calculate lower bound for game B
lower_b = 1-beta.ppf(.975, 1, 20)


Game A: $$0.879 \leq \mu_A \leq 0.918$$

Game B: $$0.83 \leq \mu_B \leq 1.0$$

So from your current data you can't tell if either game is better than the other.

You could use the Fisher's exact test to test the significance of your result.

For a quick check, you can use this online calculator and plug in your numbers.

• is the Fisher's exact test only for two data? Men and Woman, A and B etc... In my case it's about hundred or thousands of data, some are just extremely large and some are small. Aug 26, 2015 at 16:04
• I didn't see that in the question. I suggest to update the question. Thanks. Aug 26, 2015 at 19:10
• There is a similiar question stackoverflow.com/questions/20499745/… and here stats.stackexchange.com/questions/1805/… Aug 26, 2015 at 19:12