How are user opinions compared? Consider a situation where people are asked to rate a product on a scale of 1 to 5.
Product A has a rating of 3.95 based on 15,000 ratings. Product B has a rating of 4.2 based on 1000 ratings.
From the data given, can we determine which product is the preferred one? Is there a direct formula to solve problems of this type? Can anyone explain in layman's terms?
PS: Please retag as appropriate. I couldn't find a relevant tag for this question.
 A: Do you not have the full distribution of ratings? What you can do with just this information is to assume the standard deviation for both A & B is 2, the most it can be on a 5-point scale, & use that to calculate the standard error of the difference between the two means.
In response to your comment: what you want to know is how well the difference in mean rating between A & B in your sample, $\Delta \bar{x}$, estimates the difference in means in the larger population, $\Delta\mu$. In large samples the difference in means will be Normally distributed to a good approximation; if you know the standard deviation $\sigma$, & having the sample sizes $n_1$ & $n_2$, you can calculate the standard error
$$\sigma_{\Delta\mu}=\sigma\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}$$
whcih gives a 95% confidence interval on the difference between means in the population: $$\Delta\bar{x}\pm1.96\sigma_{\Delta\mu}$$
Without the full data-set you don't know the standard deviation, but can use a worst-case value of two to give a conservative confidence interval, i.e. one which will cover the population value with probability of 95% or better. With the complete data-set of course you can estimate the standard deviation (& check the assumption that it's the same in A & B).
