7
$\begingroup$

I need an idea for a new rating system.. the problem in the ordinary one (just average votes) is that it does not count how many votes...

For example consider these 2 cases:

3 people voted 5/5

500 people voted 4/5

The ordinary voting systems just take the average, leading to the first one to be better.. However, I want the second one to get a higher rating, because many people have voted for the second...

Any help?

$\endgroup$
9
$\begingroup$

You are talking about a shrinkage estimator. Imdb is possibly the most famous example of this, how they calculate which movies will make it onto the top 250. It relies on the equation,

weighted rating (WR) = (v ÷ (v+m)) × R + (m ÷ (v+m)) × C , where:

* R = average for the movie (mean) = (Rating)
* v = number of votes for the movie = (votes)
* m = minimum votes required to be listed in the Top 250 (currently 3000)
* C = the mean vote across the whole report (currently 6.9)

They call this a "true bayesian rating" and that's true in the sense that our prior for the parameter "average rating" is that it is the same as for all other movies. This prior is then updated based on the "likelihood," which is the average rating for that movie, which has more strength if it has more votes. But I'm not sure whether this technically qualifies as bayesian, because neither the prior nor the posterior is a distribution... Can anyone clarify this?

$\endgroup$
6
$\begingroup$

You could use a system like reddit's "best" algorithm for sorting comments:

This algorithm treats the vote count as a statistical sampling of a hypothetical full vote by everyone, much as in an opinion poll. It uses this to calculate the 95% confidence score for the comment. That is, it gives the comment a provisional ranking that it is 95% sure it will get to. The more votes, the closer the 95% confidence score gets to the actual score

So in the case of 3 people voting 5/5, you might be 95% sure the "actual" rating is at least a 1, whereas in the case of 500 people voting you might be 95% sure the "actual" rating is at least a 4/5.

$\endgroup$
1
$\begingroup$

You may run into problems implied by the Gibbard Satterthwaite Theorem or Arrow's Impossibility Theorem, or any of the results of voting theory...

$\endgroup$
1
$\begingroup$

There is a simple (also to implement) heuristic to first seed the pool of votes with small number of dummy votes with average voting, and later replace it with incoming votes.
So, for instance, new object appears and you give it few votes rating it 2.5/5 (this is the best you can tell about it at the zero-knowledge point). Then the first vote comes, let's say 5/5, but it is somewhat tempered by the rest of the initial pool and the objects mean is slightly above 2.5. Than next votes come and the mean gradually moves from the initial guess to the real average which than has time to stabilize. Finally this algorithm converges to normal vote mean.

$\endgroup$
  • $\begingroup$ "...and later replace it with incoming votes." Just a little fix. In 'standard' bayesian average dummy votes are permanently added (and not replaced by actual votes), so (providing - as in your example - that only 5/5 votes are given) the value converges but it will never show the exact average. $\endgroup$ – Tomaso Neri Dec 3 '15 at 10:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.