I have two groups, "in" and "out," and item categories that can be split up among the groups. For example, I can have item category A that is 99% "in" and 1% "out," and item B that is 98% "in" and 2% "out."

For each of these items, I actually have the counts that are in/out. For example, A could have 99 items in and 1 item out, and B could have 196 items that are in and 4 that are out.

I would like to rank these items based on the percentage that are "in," but I would also like to give some priority to items that have larger overall populations. This is because I would like to focus on items that are very relevant to the "in" group, but still have a large number of items in the "out" group that I could pursue.

Is there some kind of score that could do this?

edit: I should add that I also know the total number of items that are in and out across all categories. I have a pretty complete picture of how many items I have, what categories they are in, and whether or not they are in/out.


You can simply calculate proportion of 'in',

$$ P = {n_{in} \over n}, $$

but - as per your question - it's better to add a correction for total number of 'votes'.

One way to correct is to add some dummy 'outs' (e.g. $10$), so

$$ P' = {n_{in} \over n + 10 }. $$ items with a large number of votes see their modified percentage alters very little from their real percentage, but items with relatively few answers will see their modified percentage move considerably toward low values.

This is known as "Bayesian averaging". In effect, the items with many votes will rank higher than items with the same percentage but fewer votes.

In your example $P'_A=0.9$ and $P'_B=0.93$.

  • 2
    $\begingroup$ Wow, this is so simple. I was trying all sorts of crazy calculations with logarithms and entropy-like formulas, but this makes much more sense and works much better. Thank you! $\endgroup$ – neelshiv Jul 28 '16 at 14:17

I don't know if there is a "standard" way to do this, but one approach is to scale the percent of each category by the number of "in" or the size of the category. For example, you could multiply the percent in by:

  • size of category (equivalent to counting the number of "in")
  • sqrt(size of category)
  • ln(size of cateogry)

You would need to select a function (one of the above or some other) that is most appropriate for your application.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.