2
$\begingroup$

So I am developing this application for rating books (think like IMDB for books) using relational database.

Problem statement :

Let's say book "A" deserves 8.5 in absolute sense. In case if A is the best book I have ever seen, I'll most probably rate it > 9.5 whereas for someone else, it might be just an average book, so he/they will rate it less (say around 8). Let's assume 4 such guys rate it 8.

If there are 10 guys who are like me (who haven't ever read great literature) and they all rate it 9.5-10. This will effectively make it's cumulative rating greater than 9 (9.5*10 + 8*4) / 14 = 9.1

whereas we needed the result to be 8.5 ... How can I take care of(normalize) this bias due to incorrect perception of individuals.

MyProposedSolution :

Here's one of the ways how I think it could be solved. We can have a variable Lit_coefficient which tells us how much knowledge a user has about literature. If I rate "A"(the book) 9.5 and person "X" rates it 8, then he must have read books much better than "A" and thus his Lit_coefficient should be higher. And then we can normalize the ratings according to the Lit_coefficient of user. Could there be a better algorithm/solution for the same?

$\endgroup$
  • $\begingroup$ You seem to be operating under the (probably) false assumption that people rate books objectively using the same criteria, and that a rating that deviates from the "expected" rating is a mistake of some kind. But people very often rate books on a completely subjective basis. "Great literature," especially so. There are those who believe that Moby Dick, for example, is a waste of time and would rate it a 2. $\endgroup$ – Jim Mischel Jul 17 '14 at 13:31
  • $\begingroup$ The identical question on StackOverflow with some other answers is here for reference. $\endgroup$ – Escher Jul 17 '14 at 14:49
  • $\begingroup$ I agree. Same thing with movies. But seems like somehow IMDB took care of the bias due to subjectivity, I wonder how :/ $\endgroup$ – suneet Jul 18 '14 at 5:23
3
$\begingroup$

To me, your question doesn't seem to be a programming question but more of statistics one as to how much to weight good opinions vs bad. Here's how I would approach it:

  1. Use a clustering algorithm to classify your reviewers according to their bias
  2. Use a classification algorithm to assign new reviewers to a cluster (eg k-NN)
  3. Take into account cluster membership when calculating your review score. Note your example assumes assymetric bias - always overestimating the true score of a book. You could have a cluster of people who randomly give reviews wide of the true score in both + and - directions, and so your weighting algorithm could be a regression that excludes this cluster. You would to have look at the residuals to see if your adjustments are appropriate (like in any regression).
  4. Look at your lift curve to see if your forecast model is reliable

Or the short answer is to just increase the number of reviewers per title and rely on the central limit theorem.

$\endgroup$
2
$\begingroup$

This boils down to you wanting to benchmark different users against one-another based on common books. The complexity comes up because there is no one book that everybody has read. The solution would be similar to Collaborative filtering. I suspect if you read up on that, the solution will immediately become clear.

$\endgroup$
1
$\begingroup$

Well, defining what it means to have much knowledge about something is hard - and measuring the knowledge is even harder.

If you have already decided to use an approach where the user's ratings have not equal weights, then I would base the weight over each user's mean rating. So if a user who normally gives ratings around 5.0 rates a book with 9 he must really like it. But if a user with only ratings of 9 rates a book with 9, it is just an average book for him.

Please note that this sketched algorithm has its own flaws (e.g. if a user only rates books he really likes) but it may be a good start.

$\endgroup$
0
$\begingroup$

My understanding of your problem : You have m (let say 10) users who have rated n(10 for now) books. In ideal scenario where each user had similar rating styles, you could simply take average of all users' ratings for every book as the rating of that book. But, different people have different rating styles.

It's difficult to suggest what would work best for you without looking at the distribution of data, however you should start with a weighted average approach. So, each user will have a weight based on the ratings that he/she has provided.

Computing weights for users:

  1. Use cosine distance formula to find distances between all the users based on their ratings. Feature vector for every user should look something like this user1 - {rating_movie1, rating_movie2, rating_movie3 ....} and so on.

  2. Now that you have distances between all the users, all you have to decide is : who should have maximum and minimum weight respectively? - For this you should look at the probability distributions (histograms) of ratings for every user. Assuming population standard deviation is not too erratic, you should be able to compute weight of every user based on cosine similarity/dissimilarity + distribution center of every user. User with highest mean should have highest weight, keeping in mind that it should have significantly different weight than users who according to cosine distance are not very similar to the particular user.

Once you have assigned weight to every user, you can compute rating for each book by taking a weighted average.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.