Timeline for Weight a rating system to favor items rated highly by more people over items rated highly by fewer people?
Current License: CC BY-SA 2.5
15 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 8, 2014 at 16:28 | comment | added | xuinkrbin. | @probabilityislogic: Trés, trés informatif! What if providing any rating is considered "success"? I presume the same logic holds, no? Also, do You have any insight on how to perform such a calculation if, instead of "like" and "dislike" and etc., the rating is any numerical value on a scale from, say, 0-9, including fractions? Does some sort of integral apply? | |
| Jan 21, 2011 at 8:10 | comment | added | probabilityislogic | Apologies, I made a small error in the above comment, the probability should be $\frac{s+\frac{cq}{m}}{n+q}$ instead of $\frac{s+\frac{q}{m}}{n+q}$. | |
| Jan 21, 2011 at 7:07 | comment | added | probabilityislogic | ...continuing again (apologies for the lengthy "comment")... Note that changing from $2$ to $q$ changes the probability to $\frac{s+\frac{q}{m}}{n+q}$. One thing I have wondered with this argument, is do we need to specify which categories within "success" are they ones we are willing to assume are possible a priori? Or can we just say "one of them, but not sure which" and then spread this out "evenly" over the aggregated categories? If we can, then this gives an interpretation of the Jeffreys prior as assuming that we consider only one outcome possible, but not sure which one or $q=1$. | |
| Jan 21, 2011 at 6:55 | comment | added | probabilityislogic | ...continuing again... So what state of knowledge does the constant $\frac{2}{m}$ represent? Here's what I think. It represents that we are only know that one category in the "success" labeled group, and one category in the "failure" group are possible, the remaining $m-2$ categories we are not even sure that these categories are possible and will only consider them possible after observing them. This easily generalizes to a new constant $\frac{q}{m}$ where there are $q$ categories possible and $m-q$ categories where we are not sure of even the possibility that they exist. | |
| Jan 21, 2011 at 6:47 | comment | added | probabilityislogic | ... continuing, this means the probability of success is given by $\frac{s+ck}{n+mk}$. So setting $k=\frac{2}{m}$ gives $\frac{s+\frac{2c}{m}}{n+2}$. Now if m is "evenly divided" as the wiki page says, then 2c=m and we are left with the original rule of succession. So what does this mean for the uniform prior (k=1)? To me, it means that mere knowledge of the existence of more than two possible categories (that we consider them possible is essential to this argument) represents important information about an equal aggregation of them. more later | |
| Jan 21, 2011 at 6:34 | comment | added | probabilityislogic | Okay, I've had a bit of a think and basically using the $Dir(k,\dots,k)$ prior to give posterior of $Dir(k+n_1,\dots,k+n_m)$. The [wiki page][1] [1]:en.wikipedia.org/wiki/Dirichlet_distribution#Aggregation shows that collapsing categories, you just add the parameters. This means that collapsing to 2 categories gives you a Beta posterior. so the denominator will be the sum of all dirichlet parameters $n_1+\dots+n_m+mk = n+mk$ and the numerator will be the sum of all the categories labelled "success" which will be of the form $s+ck$ where c=number of categories for "success". | |
| Jan 21, 2011 at 4:44 | comment | added | probabilityislogic | @onestop - I would argue against the wiki page that you describe is only considering the case when 1 of 2 possible outcomes can happen. So if we were indifferent between "strongly like" and everything else (the other three categories combined), then we are not indifferent between "strongly like" and "like". But the question as posed clearly give four categories, not two, and from the information in the question there is no a priori reason to favour any one rating. I'll have another think about it, because the issue may be more subtle (e.g. why not use the reference prior?). | |
| Jan 20, 2011 at 22:32 | comment | added | onestop | @probabilityislogic: surely any strictly positive parameters for the Dirichlet prior describe that all the probabilities are strictly between 0 and 1? And this argument suggests setting them to 2/m, where m is the number of categories, rather than 1: en.wikipedia.org/wiki/… | |
| Jan 19, 2011 at 20:14 | comment | added | Andrew | One more observation, for future people who find this post: In implementing this in my model I took the final score and multiplied it by 20, which gives a range of -100 to 100 from worst to best possible score (though I suppose technically those are limits you can't ever quite reach, but you get the idea). This makes the output for users in my app very intuitive! | |
| Jan 19, 2011 at 13:25 | comment | added | probabilityislogic | While they may seem like "fake" observations, they do have a well defined meaning when it is +1 (as opposed to +2 or higher, which really are "fake" numbers, or numbers from a previous data collection). It basically describes a state of knowledge that it is possible for each category to be voted for, prior to observing any data. This is precisely what the flat prior on the (N-1) simplex does. | |
| Jan 19, 2011 at 9:28 | comment | added | onestop | Very nice non-technical answer, and an approach I wouldn't have thought of myself. I'd only add that it's possible to add any number of fake 'observations' to each category instead of 1, including non-integer numbers. This gives you flexibility to decide how much you want to 'shrink' towards zero the scores of items with few votes. And if you happen to want an technical-sounding description of this method, you could say you're performing a Bayesian analysis of data from a multinomial distribution using a symmetric Dirichlet prior. | |
| Jan 19, 2011 at 0:11 | comment | added | Andrew | That was a bit "mathsy" for me, and initially I didn't understand the formula, but I read it carefully about three times and it clicked! This is exactly what I was looking for, and your explanation was very clear, even for someone who isn't a mathematician or statistician at all. Thank you very much! | |
| Jan 19, 2011 at 0:10 | vote | accept | Andrew | ||
| Jan 19, 2011 at 0:02 | history | edited | onestop | CC BY-SA 2.5 |
close unmatched "
|
| Jan 18, 2011 at 23:42 | history | answered | probabilityislogic | CC BY-SA 2.5 |