I had survey respondents rank a series of items in order of importance, from 1 to 7. So, if a respondent assigned a score of 1 to one variable, then none of the other variables could get a 1 as well. The dataset is replicated in test.data below. I have three questions:

  1. Should I interpret this as ranked data rather than interval data? I think so, but I am not sure.
  2. If they are ranked data, is it sensible to use either the Wilcoxon-Mann-Whitney U or Kruskal-Wallis test to test for differences in median (?) rankings depending on the levels of the predictor variable?
  3. If they are ranked data, could I construct a correlation matrix using Spearman's Rho?
  4. If that is possible, could I use a factor analysis on that correlation matrix to possibly reduce the dataset and measure some hypothesized underlying constructs?

I have tried to do these steps basicallly using the steps below, although I see now that there is a paper and an R package suggesting the possibility of performing factor analysis on ranked data.

R Package For Ranking Data Yu, Lam and Lo 2005

Thank you for your suggestions.

#Create Data frame
test.data<-replicate(10, sample(seq(1,6,1), replace=F))


#data frame

#Provide names
names(test.data)<-c('item1', 'item2', 'item3', 'item4', 'item5', 'item6’)

#Some predictors
test.data$gender<-factor(sample(c('M', 'F'), replace=T,size=10))
    test.data$position<-factor(sample(c('Journ', 'Pol'), replace=T, size=10))

#Correlation Matrix
cor.matrix<-cor(test.data[,1:6], method=c('spearman'))

#Factor Analysis
plot(eigen(cor.matrix)$values, type='o')

fa(cor.matrix, method='pa', rotate='none', nobs=nrow(test.data))

#Kruskal-Wallis or Mann-Whitney depending on predictor levels
lapply(test.data[,1:6], function(x) kruskal.test(x~position, data=test.data))
  • $\begingroup$ The first link declines opening. Do you have the full article which you can share? $\endgroup$
    – ttnphns
    Mar 13, 2015 at 20:38
  • $\begingroup$ [link]ncbi.nlm.nih.gov/pmc/articles/PMC3665468 $\endgroup$
    – spindoctor
    Mar 13, 2015 at 21:06
  • $\begingroup$ Pardon me, I meant to have the second article "Factor analysis...". Do you have a pdf that you could share? $\endgroup$
    – ttnphns
    Mar 14, 2015 at 7:32
  • $\begingroup$ JSTOR Shared From Google Drive $\endgroup$
    – spindoctor
    Mar 20, 2015 at 14:18

1 Answer 1

  1. Yes. However, not very seldom rankings are treated as interval data and are analyzed by parametric procedures (such as ANOVA). For example, this is customarily done in classic conjoint analysis. Conceptually, psychometrically, rankings are ordinal while ratings are potentially interval (albeit often cautiously treated as ordinal as well). Still, statistically both are just numbers on a quantititative scale, therefore unless specifically statistical assumptions are strongly violated they could be processed in the same manner. Especially by a univariate analysis. Note that if somebody considers each of the 6 variables that you generated in your example separately he will not be able to say whether they are “rankings” or “ratings”.
  2. Yes. However, those tests are sensible with interval data either.
  3. Yes. However, Spearman is sensible with interval data either, to capture nonlinear monotonic relationship.
  4. No. Classic (linear) factor analysis is only for Pearson correlation (and similar SSCP-type measures, read also). Spearman is based on nonlinearly transformed values, ranks (not your original "ranks", rankings - but the ones the procedure internally produces). Linear FA will erroneously "think" that those transformed values are the original values or are linearly transformed original values and will "uncover" linear underlying constructs which actually are not linear or even do not exist in your actual data. So use Pearson - if you dare to see the rakings data as ratings data. Alternatively, there exist special factor analytic procedures for categorical data, such as factor analysis on polychoric correlations, IRT factor analysis, PCA with optimal scaling (CATPCA).

Whatever factor analysis or other multivariate analysis you do on the rankings data you should be aware that the ordered multinomial (no ties) nature of ranking task induces negative correlations in the the data. In your code, for example, you generate 6 variables which are random ranking from 1 to 6. Expected correlations between the variables will be all -1/(6-1) = -0.2. (See also, about compsitional data in general>) If these were ratings, the expected correlations would be all 0. I’m speaking of the baseline or background correlations – expected in the absence of substantive factors of interest. The presence of a background general factor does not preclude doing factor analysis, generally, but it may be a complex issue – how to deal with it the best way.

  • $\begingroup$ 1. Really? OK, I did not know that. So, I could get away with a linear regression or ANOVA to test for difference in group means in the rankings for each item? 2. OK, good to know. 3. OK, thanks 4. OK, but If I'm going to conceive of them with interval data, then presumably I could construct a Pearson correlation matrix to send that to a factor analysis, no? $\endgroup$
    – spindoctor
    Mar 13, 2015 at 21:08
  • $\begingroup$ In point 1, I didn't say the decision to treat human rankings as interval rating data is just flawless. I said, not seldom it is done. For example, in classic conjoint analysis. ANOVA and other similar parametric analyses are robust unless the distribution of errors is considerably skewed. So, inspect errors in your real data. Point 4 - yes. But, again, people might frawn at your decision to treat rankings as ratings. $\endgroup$
    – ttnphns
    Mar 14, 2015 at 7:46
  • $\begingroup$ Re: point 4: I know polychoric correlations. I'm pretty comfortable thinking of these as rankings, rather than ratings, so I might proceed with polychoric rankings > factor analysis > regression of resulting factor scores on predictors. thanks, very much! $\endgroup$
    – spindoctor
    Mar 20, 2015 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.