Thanks in advance for bearing with me, I am not a statistician of any kind and don't know how to describe what I'm imagining, so Google isn't helping me here...
I'm including a rating system in a web application I'm working on. Each user can rate each item exactly once.
I was imagining a scale with 4 values: "strongly dislike", "dislike", "like", and "strongly like", and I had planned on assigning these values of -5, -2, +2, and +5 respectively.
Now, if every item was going to have the same number of ratings, then I would be quite comfortable with this scoring system as clearly differentiating the most liked and least liked items. However, the items will not have the same number of ratings, and the disparity between the number of votes on different photos may be quite dramatic.
In that case, comparing the cumulative scores on two items means that an old item with a lot of mediocre ratings is going to have a much higher score than an exceptional new item with many fewer votes.
So, the first obvious thing I thought of us to take an average... but now if an item has only one rating of "+5" it has a better average than an item that has a score of 99 "+5" ratings and 1 "+2" rating. Intuitively that isn't an accurate representation of the popularity of an item.
I imagine this problem is common and you guys don't need me to belabor it with more examples, so I'll stop at this point and elaborate in comments if needed.
My questions are:
- What is this kind of problem called, and is there a term for the techniques used to solve it? I'd like to know this so I can read up on it.
- If you happen to know of any lay-friendly resources on the subject, I'd very much appreciate a link.
- Finally, I'd appreciate any other suggestions about how to effectively collect and analyze this kind of data.