Netflix used to base its suggestions on a user's submitted ratings of other movies/shows. This rating system had five stars.

Now, Netflix allows users to like/dislike (thumbs-up/thumbs-down) movies/shows. They claim it's easier to rate movies.

Wouldn't this 2-way classification be statistically less predictive than a 5-way classification system? Wouldn't it capture less variation?

  • 1
    $\begingroup$ I think any answers should take seriously Netflix's own explanation, including that in A/B testing they got 4x as many ratings in the 2-way system. That alone is huge. Also, the upshot of the famed Netflix competition was that better predicting ratings (what they got from the competition) didn't translate into better predicting behavior. You might judge "Pride and Prejudice" a better movie, but be more likely to watch "Die Hard". Netflix probably cares a lot more about what you want to (or will) watch than what you think is a better movie. $\endgroup$ – ctwardy Apr 19 '17 at 14:01

According to an article by Preston & Coleman (2000), 2 item-scale relaibility does not differ markedly from 5 item-scale reliability:

Reliability of item-scales

The subject of measurement was satisfaction with restaruants but it translates well to the movie rating. Ease of use, how quick it is to use and how well can a person express feelings on a differnt item-scales was measured as well. The results are as follows:

Satisfaction measures

It is clear that users find 2 item-scale slightly easier to use and quicker to use in comparison to 5 item-scale but also very inadequate in expressing user's true beliefs. This indicates that 2 item-scale does not capture underlying variability very well and results in loss of variability. Discrimination indices are also markedly poorer for 2 item-scales in comparison to 5 item-scales.

Taking all of the above into account I would speculate that Netflix is willing to exchange some voting precision to lure more users into voting. I think they prefer more people voting since it increases sample coverage. This can lead to better understanding of less engaged users. Marginal value of additional information for less engaged users is likely much higher in comparison to engaged users.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.