2
$\begingroup$

I have a collection of products that could be rated as good or bad. Every product has a grade, that is the sum of "good" votes minus the sum of "bad" votes. These products are also split into two categories. What I want to do is compare both categories and see which category has better product grades.

This would be simple if all products had the same popularity. We have extremely popular products with hundreds of votes and not-so-popular products with less than 10 votes. It's clear that ten "good" votes have different impacts on the final grades of each product depending on its popularity.

One information that could help me to consider the product's popularity would be the number of views each product received. More visits = more popular. But what I don't know is how to use this information to help me compare the products equally. Is using its grade divided by its popularity (views) enough ?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I suggest a logistic regression model, defined over individual judgements rather than the aggregate score. Response variable is whether the vote was good or not. Predictors are product id and category.

The test is a standard test for significance of a predictor in a logistic regression model for the category factor. For example, compare the two models using a chi squared test (full model vs model without category factor).

Edit: if @Scortchi is correct and each product only occurs in one category (which makes sense), then his suggestion is correct.

The reason you want to include a predictor for product is that different products are presumably of different quality and so there's variation between products irrespective of which category they're assigned to. If you don't capture this in your test you're missing part of the information and your test will have less power.

I suggest using a regression model because many tests on more complex data with multiple factors can be viewed as tests on regression models. If you do as I say and treat each judgement as a data point instead of each product, you'll naturally incorporate the frequency bias you want (more popular product = more data points) in a principled way, instead of testing against a derived and I would argue flawed aggregate metric.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I'd guess each product ID occurs in one category only, so you wouldn't be able to include both it and category as fixed-effect predictors: probably best to model each ID as a random intercept, or use a beta-binomial model, or at least use a quasi-likelihood method to allow for over-dispersion. $\endgroup$
    – Scortchi ♦
    Commented Apr 10, 2014 at 14:10
  • $\begingroup$ I don't understand why I should use the product's ID in the model. What I want to know is if products of category A are better than products of category B using the grade and it's popularity (views). I'm not sure either if a regression would be the best way to compare both categories .. $\endgroup$
    – Milena Araujo
    Commented Apr 13, 2014 at 16:37
  • $\begingroup$ What I want is a way I could say that, for instance, this product with grade 10 and 100 views is as good as this other product with grade 100 and 10000 views, based on the data I have. So then I could fairly compare the products without caring about the popularity bias. $\endgroup$
    – Milena Araujo
    Commented Apr 13, 2014 at 16:40
  • $\begingroup$ Now I got it, Ben. Thanks for the explanation. My only problem following your suggestion is that I don't have individual judgment for each product, only the final grade was stored. Any thoughts on this ? $\endgroup$
    – Milena Araujo
    Commented Apr 14, 2014 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.