1
$\begingroup$

I have a dataset where I'm comparing overall ratings between multiple "products." It looks something like this:

Product 1: 3.5

Product 2: 4.1

Product 3: 3.7

...

Product 7: 3.3

What I'm doing is comparing Product 1 to all the others and I'd like to highlight whether or not there's a statistical difference in the ratings (if that's a thing) between Product 1 and the others. I'm not sure if that's possible.

I read that maybe using an independent t-test is the way to go, but I'm not sure if that's 100% accurate since the dataset I have isn't all from the same raters. For example, the data on Product 1 wasn't rated by the same people as Product 2.

Is there a more accurate way to do this?

$\endgroup$
8
  • $\begingroup$ Do you have the individual ratings? $\endgroup$
    – Dave
    Nov 19, 2021 at 17:47
  • $\begingroup$ Is all you have the per-product summary rating? You won't be able to do statistical analysis without knowing anything about the sample size. If a single person rated Products 1 and 2 as a 3.5 and 4.1, you can't make any statistical inference at all. If a handful of people rated them this way, it's probably not significant, but If thousands of people independently rated them in that manner, it likely will be. You won't be able to do much with just the summary ratings. $\endgroup$ Nov 19, 2021 at 17:49
  • $\begingroup$ @Dave I don't - just the final results/ratings of each. $\endgroup$
    – DD1229
    Nov 19, 2021 at 17:49
  • $\begingroup$ How were you planning to do an independent t-test? $\endgroup$
    – Dave
    Nov 19, 2021 at 17:52
  • $\begingroup$ @NuclearHoagie - yeah, I just have the summary rating. We can assume that there were thousands of people rating the product, but we don't have the per-person ratings, just the overall. $\endgroup$
    – DD1229
    Nov 19, 2021 at 17:52

2 Answers 2

0
$\begingroup$

Without having the individual measurements, I cannot see a way to do this as an ANOVA-style comparison between groups. However, analysis of your data is not impossible. The way I might proceed is to give the relative positions of the products. Of the four you listed, product $1$ is middle-of-the-pack.

Particularly if you have more than seven products (preferably more like $>100$), you can be formal about this by giving the percentiles of the products. Wouldn't it be nice to tell your boss that product $1$ is in the bottom $10\%$ of ratings but product $90$ is in the top $2\%$ of ratings?

$\endgroup$
1
  • $\begingroup$ That's a really good idea! $\endgroup$
    – DD1229
    Nov 19, 2021 at 19:35
0
$\begingroup$

If the average ratings you have are based on thousands of responses, it is virtually certain that the differences are statistically reliable (i.e., statistically significant) and not due to chance or random fluctuation. The question then is are these differences practically meaningful. I suggest you use a two prong approach to reach an opinion on the question of meaningfulness of the differences in which you have an interest. First, look at the rating scale. What are the definitions of the rating scale anchors for 3 and 4. If one anchor is "acceptable"and the other "less than acceptable" that gives some meaning to the means of 3.5 and 4.1. Second, if you know anything about any two of the products that were rated, you might be able to say that the difference between these two known products is this big, so any other difference of that size has some meaning in terms of what you know about the two known products.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.