6
$\begingroup$

After reading "How Not to Sort by Average Rating" (http://www.evanmiller.org/how-not-to-sort-by-average-rating.html), I was curious to know if there was the same thing for variables with more than two outcomes (0,1) or even continuous variables.

For example, how would you generalise the lower bound to the Amazon problem ? Clearly there are 5 outcomes (one for each number of star given by the user). What measure would you use to make the 4.5 stars with 2000 votes better rated that the 5 star with 2 votes ?

Also, it seems to me that this kind of problem could have a bayesian interpretation. I mean using the formula in "How not to sort" is not far from setting a prior in the distribution, maybe a Bernoulli with parameter inferred on the whole dataset / category the item belongs to ? Does anyone know a reference for this particular problem ?

$\endgroup$
0

1 Answer 1

2
$\begingroup$

It's easy to think of the following 'workaround' which adapts a multi-ranking system to the 'upvote/downvote' solution discussed in the linked article:

Let's say you have the popular 5 star rating system. So we have a number of votes, each having a value of: 1, 2, 3, 4 or 5.

To 'convert' these ratings to up/down votes, use the following rule:

For star rating -- Add

*     - 0.00 to up votes and 1.00 to down votes (i.e. a full down vote)
**    - 0.25 to up votes and 0.75 to down votes
***   - 0.50 to up votes and 0.50 to down votes
****  - 0.75 to up votes and 0.25 to down votes
***** - 1.00 to up votes and 0.00 to down votes (i.e. a full up vote)

After we reduce the 5 star ratings to up/down ratings, we can proceed with the usual score calculations described in Evan Miller's article.

As I am not a statistician or mathematician and I would love to hear from other people if this makes sense or not.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.