I came across an interesting method of ranking entities (products) today, and I wanted to hear some perspectives on this approach. Basically, the assessment asks a set of 30 personal value statements of respondents (i.e. social responsibility is important to me) rated on 5pt Likert scale, these are reduced through Factor analysis to 3 dimensions. Then respondents are asked to rate 50 different products using a single 10pt Likert scale for each product. How much do you like x product? Single variable, 50 different products. So to compute their "score" for a single product...
the top 1/3 of scorers on each factor are counted.
Then, of those, only those that scored their liking of a given product really high (i.e. 8,9,10) are counted. So you might have 100 respondents, 10 in the top 1/3 and only 5 scoring a given product high. This is repeated for each factor. So factor 1 might be .5, factor 2 might be .21, and factor 3 might be .35. it's just the proportion.
Wait there is more. So to now weight each of those ratios, they perform a linear regression of the factor scores for each participant on the specific product likeness score ( remember the likeness score is a single variable) and then use the resulting coefficients to weight those ratios and then add them together. So .5*coef1+.21*coef2 +.35*coef3=y. Then y is rescaled to between 1and 100. This process is repeated for all 50 products to produce essentially a ranked set of products.
So... That's not how I was taught to do indexing and ranking way back in grad school. It smells like Net Promoter Score, which suffers from a whole lot of methodological problems. My main complaints are: 1. I don't think this is valid at all 2. They are using only a fraction of the respondents in computing the "score" 3. The way they are using regression to weight the proportions is weird, and I think wrong. 4. I don't think that the resulting product scores can really be compared using this approach.
In my experience, you would create an index of a set of variables (way more than 1) for each product, ideally using factor analysis to reduce the variables to a single value. It's as if the whole methodology they describe is backwards.
However, perhaps this is standard procedure in marketing?
Just thought I would ask what other professionals have encountered.