How to analyse a ranking and rating scale together?

Question

For a market research; consumers are asked to rank the features of a product based on the priority. For example,

Rank the following features for a device based on your priority (1 being the top most priority)

Storage capacity         6
Portability              5
Touch interface          1
Keyboard                 4
Long battery life        2
Entertainment on the go  3

On a scale of 1 to 5, rate the features based on the importance (1 being very important)

                    1   2   3   4   5
Storage capacity    1               
Portability                 3       
Touch interface     1               
Keyboard            1               
Long battery life       2           
Entertainment on the go         4   

Now based on the ranking and rating, I want to assign weights and in the end find out whether the consumer prefers a laptop or a tablet pc

Say in this example the consumer has ranked touch as 1 and battery life as 2, these are the attributes of a tablet. But he has rated keyboard and storage capacity as the most important, which are the features of a laptop with a keyboard.

How do I combine both these and probably assign some weights and come up with a score. If the score is above a certain level then the consumer prefers a tablet and below a certain level then the consumer prefers a laptop.

Answer 1

I assume you can't just ask them if they prefer a laptop or a tablet; or you want to check what they think they prefer with what you think they should prefer...

There are a number of ways to do this. This is in fact a version of the very common real life problem of evaluating job applicants, or tenders for contracting work - you need to decide on criteria, weight them, and rate the candidates against the criteria. You have emphasised the problem of weighting the criteria, but the rating of candidates (laptop and tablet) against the criteria is crucial, as was the choice of the six criteria in the first place. These are largely judgement rather than statistical questions.

There are two steps necessary: combine the information in the two questions to give you weightings for the criteria; and the compare the importance given to the six qualities to the performance of the two products against those six qualities.

Your first problem is that you have two questions that are apparently (see my comment) getting at basically the same underlying factor and that respondents will inevitably not be completely consistent in their answers (although hopefully not as much as in your example, where storage capacity is the lowest priority but "very important"!)

One approach to combining these two is to convert the ranking to a rating on the same scale as the second question and then take an average. You could do this for example by $rate_{new}=\frac{rate+rank*\frac{4}{5}+0.2}{2}$.

This is a bit crude, but the fact is there is no really satisfactory way of combining the two without drawbacks of some sort. Converting rankings to ratings and vice versa is a problem however you do it, and some kind of rule of thumb is needed to deal with ties in the ratings (if you want to turn them into rankings) or unknown range behind the rankings (if you want to turn them into ratings ie the user has been forced to rank from one to 6, but really might think they are all really important - or unimportant...).

The next crudity is you will need to score the products against the six qualities. Often subjects would have been asked to do this, but in this case it looks like you have to do it yourself. You will produce a matrix like:

                        Tablet   Laptop
Storage capacity         4          2
Portability              1          2
Touch interface          1          4
Keyboard                 5          1
Long battery life        3          2
Entertainment on the go  1          3

I've kept to the convention you have of low scores being good.

Then you just multiply and sum your importance ratings by these quality scores and you get a score for tablet and one for laptop. The one with the lowest score is the preference - you don't need a threshold, just to compare the two scores.

Note that how you score the two products against the six qualities will be crucial in this - probably more important than how you generated the weightings. So you'd want to try a range of different scores and see which ones give plausible results. There's no statistical way of getting the "right" scores, with the information you've got. If you knew people's actual laptop/table preferences, you could perhaps generate a set of scores that produced those preferences, but then the whole exercise would be a different one.

See below for some R code and output that implements this and suggests that your somewhat confused subject might actually want a laptop:

> r1 <- c(6,5,1,4,2,3)
> r2 <- c(1,3,1,1,2,4)
> newrate <- (r2+r1*4/5+.2)/2
> products <- as.matrix(data.frame(Tablet=c(4,1,1,5,3,1), Laptop=c(2,2,4,1,2,3)))
> cbind(products, newrate)
     Tablet Laptop newrate
[1,]      4      2     3.0
[2,]      1      2     3.6
[3,]      1      4     1.0
[4,]      5      1     2.2
[5,]      3      2     1.9
[6,]      1      3     3.3
> newrate%*%products
     Tablet Laptop
[1,]   36.6   33.1

Answer 2

To think that you can learn what is important in people's decisions simply by asking expresses unjustified optimism. But there are some sound methods of "deriving" the importance of different factors. Years and years of research in psychology and behavioural economics have borne this out. A colleague and I summarized some findings from the literature on this topic and explored some ways to apply them (in a higher education context) here.

Answer 3

This is an unsupervised learning task. Here is a very simple idea which if incorrect I hope someone else points out. Feed your ten variables into a PCA to extract 2 PCs. Use the two principle components in a 2-mean clustering algorithm to define boundaries for the assignment to each of two groups. Examine the PCs, and name them Tablet and Laptop if it makes sense to do so. You now have a criterion based on a linear combination of your 10 variables.

The main problem I see with this is that you won't necessarily end up with a definite "laptop" versus "tablet" prediction algorithm. To get something like that you would ideally have at least a few data points with outcomes to train on.