1
$\begingroup$

I am figuring out how to build a ranking system for movies using IMDB data. My own dataset related exclusively to "art movies", so they would tend to have fewer reviews than the average IMDB movie (the mean is 3,000 for my own dataset). In particular, I've considered using the IMDB "Bayesian estimator" for its Top 250 list (shown below). The problem is that I don't understand how the $m$ (minimum votes) constant is calculated. I read somewhere that it is a rather arbitrary constant but it could be thought as the minimum number of reviews needed to "expect" a rating similar to a "true rating" (assuming all reviewers actually rated a given movie). But it is not entirely clear to me how to think about this $m$ constant for my own case.

Would choosing $m$ to be the average number of votes in my dataset a valid choice?

There are a lot of interesting discussions regarding Bayesian ratings here on CV, which themselves link to (what looks like it could be) great reading material to better understand Bayesian ratings (for example, a blog entry by Andrew Gelman), but most of the links are dead, so I can't read them.

Could you please recommend any good material on Bayesian ratings for me to get a better grasp of how to build one for my own? I have a good grasp of statistics and basic Bayesian probability, but I would prefer introductory material.

Weighted rating (WR) = (v ÷ (v+m)) × R + (m ÷ (v+m)) × C , where:

* R = average for the movie (mean) = (Rating)
* v = number of votes for the movie = (votes)
* m = minimum votes required to be listed in the Top 250 (currently 25,000)
* C = the mean vote across the whole report (currently 6.9)
$\endgroup$
2
$\begingroup$

The Bayesian rating formula looks very simple to me.
Let me break it down and present an alternate view of the parameters involved.

WR = $\frac{v}{v+m}*R+\frac{m}{v+m}*C$
=P(enough evidence)*[Rating based on evidence]+P(no evidence)*[best guess when no evidence]

Say, a movie got v ratings and you have empiracally chosen m based on what you think is a good number of votes to rely on the rating based on votes.
Observe that the second term ($\frac{m}{v+m}*C$) in the sum goes to zero when v is large. The best guess for the rating, when there is no evidence is the average rating of all the movies.

I think choosing m as the average number of votes in the dataset is over-kill. I would start with the first quartile of the number of votes.

Thanks

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks, this is really helpful. Could you add a literature reference or blog post to go with the answer so I can mark it as resolved? I would appreciate even an introductory Bayes probability book that includes a chapter on how to use Bayes for ranking. Also (this is beyond the scope of my original question), do you have any ideas on how to incorporate a measure of variability to this estimator? Say, taking into account user's ratings for a given movie are more polarized (1 5 star review and 99 1 star reviews) than others? $\endgroup$ – user3824382 Jan 7 '16 at 13:24
  • $\begingroup$ Section 1.2.3 in "Pattern Recognition and Machine Learning" by M. Bishop, the weighted rating equation above is a direct application of the formula 1.45. About your query on variability, you can just use the rating variance to establish the confidence interval on the weighted rating. For example, when the ratings are 1 5-star and 99 1-star then the ratings will have very small variance and hence will give a tight confidence interval on the weighted rating. Just read up on Central Limit Theorem and Gaussian Distribution. $\endgroup$ – Vihari Piratla Jan 7 '16 at 13:45
  • $\begingroup$ If I use the 1st quartile of the # of votes for m (~330 votes), some of the movies have very few in comparison (3 or 6 votes) as these are "art movies". Basically, the Bayes estimator would tend to rely much more on the avg movie rating and almost not at all on the actual evidence. These seems correct, what do you think? But I wonder if there is any way to account for the fact that art movies tend to have very low viewership and that doesn't necessarily mean they have unreliable ratings. Maybe setting m very low? But then, as m approaches 0, the weighted avg and the actual avg are the same. $\endgroup$ – user3824382 Jan 7 '16 at 15:48
  • $\begingroup$ Every model has its own restrictions. If you try to formulate the model on paper, you will make it unnecessarily complex. I suggest you try the weighted rating first and make the model complex only if it does not serve your purpose. One way is to handle "art movies" separately with a lower m. $\endgroup$ – Vihari Piratla Jan 8 '16 at 8:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.