I came across an interesting method of ranking entities (products) today, and I wanted to hear some perspectives on this approach. Basically, the assessment asks a set of 30 personal value statements of respondents (i.e. social responsibility is important to me) rated on 5pt Likert scale, these are reduced through Factor analysis to 3 dimensions. Then respondents are asked to rate 50 different products using a single 10pt Likert scale for each product. How much do you like x product? Single variable, 50 different products. So to compute their "score" for a single product...

the top 1/3 of scorers on each factor are counted.
Then, of those, only those that scored their liking of a given product really high (i.e. 8,9,10) are counted. So you might have 100 respondents, 10 in the top 1/3 and only 5 scoring a given product high. This is repeated for each factor. So factor 1 might be .5, factor 2 might be .21, and factor 3 might be .35. it's just the proportion.

Wait there is more. So to now weight each of those ratios, they perform a linear regression of the factor scores for each participant on the specific product likeness score ( remember the likeness score is a single variable) and then use the resulting coefficients to weight those ratios and then add them together. So .5*coef1+.21*coef2 +.35*coef3=y. Then y is rescaled to between 1and 100. This process is repeated for all 50 products to produce essentially a ranked set of products.

So... That's not how I was taught to do indexing and ranking way back in grad school. It smells like Net Promoter Score, which suffers from a whole lot of methodological problems. My main complaints are: 1. I don't think this is valid at all 2. They are using only a fraction of the respondents in computing the "score" 3. The way they are using regression to weight the proportions is weird, and I think wrong. 4. I don't think that the resulting product scores can really be compared using this approach.

In my experience, you would create an index of a set of variables (way more than 1) for each product, ideally using factor analysis to reduce the variables to a single value. It's as if the whole methodology they describe is backwards.

However, perhaps this is standard procedure in marketing?

Just thought I would ask what other professionals have encountered.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.