How to summarise the ordering pattern of a set of four stimuli by participants? I did an experiment where I asked 13 participants to order 4 stimuli (let's call them stimulus A, B, C and D), so there are 24 possible ways to order 4 items.
Now I would like to know how to analyze this data in the correct way.
Indeed I would like to say that on average a certain order configuration has been preferred.
For example I would like to say that considering all the configurations
expressed by a participant, the following represents better the order preference of the participants


*

*stimulus D

*stimulus B

*stimulus C

*stimulus A

 A: One approach would be to think in terms of pairwise comparisons and apply the Bradley Terry model.  A person who ranks the items as D,B,C,A would be equivalent of saying D beats B, D beats C, D beats, A, B beats C, B beats A, and C beats A.  Then another person with a different order would result in a different set of pairwise comparisons.  You would then combine all the pairwise comparisons and get a set of rankings of the 4 choices. 
A: You would want to convert the orderings into a categorical variable that takes 24 values, and analyze that variable. The test whether all combinations are equally likely will be given by a Pearson test with the null that all proportions are equal to 1/24. (By a rule of thumb of having an expected count of at least 5, you need at least 120 total participants.) For fancier analysis, you can formulate this as a multinomial logit model, again with 24 categories. (You would have to have all the possible combinations in the data set for this to work out.) An intercept-only model your model will tell you the relative probabilities (or rather log odds ratios against a selected baseline alternative), and you can test that all the coefficients (plus the baseline of zero) are smaller than the selected alternative. This is a difficult test per se, mind you, since you are talking about a multivarate one-sided alternative, so the distribution of the test statistic is a non-standard sum of $\chi^2$ with different degrees of freedom. (I am not sure you even want to mess with that; an extremely conservative approximation is to use $\chi^2_{23}$ or whatever the degrees of freedom of the test would come to be.)
Note that if you first pick the most frequently preferred combination in your data, and then test against, it would be data snooping with no control over type I error. The only way this test would work if your most preferred ordering came from some sort of substantive consideration (e.g., brandy is better than wine, which in turn is better than beer, which in turn is better than milk).
