Analyzing multiple, within-subjects, non-independent ordinal rankings

Question

So I'm working with some previously collected data where participants (N=139) were asked to rank 3 types of events (1 being best, 3 being worst) for 6 separate scenarios. The question of interest is whether, across all 6 scenarios, the 3 types differ in their rankings. Is Type A ranked higher than Type B? Is Type A ranked higher than Type C? Is Type B ranked higher than Type C?

I'm not sure how best to conceptualize and analyze this data. Obviously, a participant's rankings for each event type for each scenario is non-independent, so how can I compare the rankings when they are dependent on one another? I'm mainly looking for an idea of what tests to look into. I've checked out Friedman's Test, but I'm not sure my data apply due to the non-independence of the rankings and the fact that the rankings are repeated across scenarios. I also can see how the data would fit into a multilevel model (rankings within scenarios within participants), but I'm not sure how to deal with the non-independence of the rankings.

As a note, the data is really easy to present DESCRIPTIVELY and leads to clear conclusions, but I'd like to be able to back-up these conclusions up with some inferential statistics if possible.

Average Rank Collapsed Across Scenario

Type 1: 1.55 (SD = .29) Ranked 1st 57%. Ranked 2nd 33%. Ranked 3rd 9%.

Type 2: 1.75 (SD = .29) Ranked 1st 38%. Ranked 2nd 53%. Ranked 3rd 10%.

Type 3: 2.70 (SD = .31) Ranked 1st 5%. Ranked 2nd 14%. Ranked 3rd 81%.

Answer 1

I know this isn't your data, but as a side note, asking subjects for rankings is generally a poor way to ask a question (depending on what you're looking for) because you are putting yourself in a situation where your data fits with a repeated measures ANOVA, but you have a subject main effect of 0 (all subjects will have the same mean). That means that your error term (the subject by factor interaction) is as large as it can be. This is one of the few cases where within subjects designs have less power than between subjects designs. Asking for ratings would give you more power.

Anyway, you'll be fine with a repeated measures ANOVA. A) most nonparametric tests are the equivalent to doing an ANOVA/regression on rank transformations of the data. Instead of asking for ratings and then transforming them yourself, you implicitly asked your subjects to do it for you. B) Even if you asked for ratings instead of rankings, your data would still be dependent. You remove the dependence by analyzing within subjects. To test the hypothesis that A is ranked higher than B which is ranked higher than C, you just test the within subject contrast that would equate to that prediction.

Edit: When formulating the answer, I didn't consider that there are only 3 things that can be ranked. That is rather few and you could run into other problems besides dependence. However, I ran the following simulation, which revealed no effect on type I error rate.

set.seed(555)
ps<-numeric()#initialize
for(i in 1:5000){
    a<-cbind(c(1,2,3),c(1,3,2),c(2,1,3),c(2,3,1),c(3,1,2),c(3,2,1))#all possible combinations
    b<-sample(c(1,2,3,4,5,6),100,T)#create a sample of 100 with equal probability
    dat<-stack(data.frame(a[,b]))#convert to one column
    dat$version<-rep(c("A","B","C"),100)#add version/type column    
    ps[i]<-summary(aov(values~version+Error(ind/version),data=dat))[[2]][[1]][["Pr(>F)"]][[1]]#get p value
}
length(ps[ps<.05])/5000
#> .0494