2
$\begingroup$

I have a a set of films, and for each films, a set of reviews - varying between 1 review and several hundred reviews for each film. Each review has a star rating from 1 to 5.

I am using Wilson's confidence interval for a Bernoulli parameter to estimate whether the film is likely to be good or not, taking into account the number of ratings (I just count any 3+-star reviews as positive, anything else as negative).

However, I'd also like to figure out how likely to be divisive a film is, given the number of ratings.

So a film with 200 reviews - 100 1-star reviews and 100 5-star reviews - is more likely to be divisive than a film with 2 reviews = 1 1-star review and 1 5-star review. However, both films clearly have the same standard deviation of ratings.

I don't think I can use the same Wilson's confidence interval calculation that I'm using for 'goodness', since the ratings aren't Bernoulli in nature (EDIT: I'm assuming they are normal).

Does anyone have any ideas on how to measure 'divisiveness' in this way?

$\endgroup$
7
  • $\begingroup$ Could you please tell us what you mean by "divisive"? Re the edit: because on the face of it no distribution of just five values can reasonably be termed "normal," what exactly are you assuming? Perhaps that they are binned values of some unobserved underlying normal distribution? $\endgroup$
    – whuber
    Commented Nov 30, 2012 at 17:33
  • 1
    $\begingroup$ Divisive: a film that everyone rates at three stars is not divisive, but a film that half of people rate as 1 star and half of people rate as 5 star is divisive. (Basically, the larger the standard deviation, the more divisive the film.) $\endgroup$
    – Richard
    Commented Nov 30, 2012 at 18:27
  • 1
    $\begingroup$ Re the edit: okay, sorry if I am wrong. Basically, they are not binomial, so I need to find a different way to look at the problem. $\endgroup$
    – Richard
    Commented Nov 30, 2012 at 18:28
  • $\begingroup$ Would this be a fair interpretation? "Given a set of reviews (assumed to be a random sample of all film viewers), what is the probability that the population standard deviation exceeds some particular fixed threshold chosen as a marker of 'divisiveness'?" $\endgroup$
    – whuber
    Commented Nov 30, 2012 at 20:09
  • 2
    $\begingroup$ People have asked questions about how to find the best or worst items based on star rating. Unless I'm missing something though, I don't think they have asked about how to find the most divisive items, though, which is why I started a new question. Perhaps I'm wrong, in which case please do point me to the relevant question. $\endgroup$
    – Richard
    Commented Dec 1, 2012 at 17:17

1 Answer 1

2
$\begingroup$

What you calls divisiveness seems analogous to what in ecology (and other fields) is called diversity, as in biodiversity. There are many possible measures, much used ones is entropy and the Simpson index. There are many posts on this site, some is

Now, the usual diversity indices takes the categories (in ecology, species) as categorical and treats them all the same. Applied to your application of "diviseveness" or "disagreement" it would treat probability vectors $(1/2,1/2,0,0,0)$ the same way as $(1/2,0,0,0,1/2)$, which is not appropriate. So we need to take into account distances among the rating scores, 4 and 5 being more similar that 1 and 5! That means to treat the rating scores as a metric space, and we can use the metric $$ d(i,j)= \vert i-j \vert $$
and use the ideas outlined at How to include the observed values, not just their probabilities, in information entropy?

That can lead to (a version of) the Rao's quadratic entropy, with $$ d(i,j)= 1- e^{\vert i-j \vert} $$ as a measure of disagreement. A paper with theoretical studies of Rao's quadratic entropy. Some numerical examples, in R:

 quadent( rep(0.2, 5))
          [,1]
[1,] 0.6403669
> quadent(c(1, 0, 0, 0, 0))
     [,1]
[1,]    0
> quadent( c(0.5, 0.5, 0, 0, 0))
          [,1]
[1,] 0.3160603
> quadent( c(0.5, 0, 0, 0, 0.5))
          [,1]
[1,] 0.4908422
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.