Comparing ratings for recipes I have often wondered is, say, 50 reviews of average 4.5 better than 31 reviews of average 4.8?
Orbiting these considerations has been some idea of statistical significance.
All uncertainty threatens to be dispelled today since I decided to ask here. No doubt there is a clear answer to the question, is it possible to determine simply from the number of ratings (and their average, or even their distribution), which recipe is better? And if so, what is a quick formula to mentally evaluate this?
Edit: To provide some more clarity, a definition of 'better':
- All reviews are rated on the real interval $[0,5]$. A recipe is modelled as having some real, independent, "objective" "score" (on the same interval) that is estimated, with error, by the ratings of the recipe reviewers. It is to this "essential" score we refer when we assert one recipe is better in comparison to another. We attempt to access this essential score through models of it, such as through the average ratings, and number of ratings of that recipe.
- If the ratings (of two recipes $F$ and $G$ for the same enreciped menu item, $M$) are on the same site, and have the same number of reviews, then barring the inclusion of any additional information, recipe $F$ is better than $G$ if $avg(ratings(F)) > avg(ratings(G))$.
- If the ratings are on the same site, and have different numbers of reviews, I don't know.
- If the ratings are on different sites, then, barring the inclusion of any additional information, as per 2 and 3.
- If additional information is available, such as, for example (and readers may choose to delve as deeply as they like in this direction):
- the reliability of site $A$ versus site $B$. This is left to the reader's discretion, and it is imagined a probability value giving the "confidence of belief in the reliability of" site $A$'s ratings, that is, a measure of site $A$'s rating's fidelity to the underlying "true" value of the recipe's rating. So, if site $A$ provides very high ratings to recipe of an item $M$, but is generally thought to be unreliable signal of betterness, while site $B$ provides somewhat lower average, with much greater reliability, to a different recipe of the same item $M$, one could make a reasonable case that site B's recipe is better than site $A$'s. Any metric of site reliability and associated mechanism of computing an ordering on recipe's must reach the same conclusion.
- the distribution of reliability of the raters, that is, a measure of how closely a rater's score for any recipe, say $score(rater, recipe)$, models the recipe's underlying "objective" score.
In terms of what's required to answer this question, 3 is necessary and sufficient, and 5 is unnecessary and (alone) insufficient, although welcome.