You can rank ordinal distributions by means of an intuitive dominance criterion: the answers to one question are better than the answers to another when it is more likely than not that a randomly chosen answer to the first will be better than a randomly chosen answer to the second.
In more detail: put all the answers to question $X$ into one hat and all the answers to question $Y$ into another hat. Draw one answer from each hat at random. We will compare these answers, which we can do because they are on an ordinal scale. Let's also agree to resolve any ties by flipping a fair coin. Let $p(X,Y)$ be the probability that the answer to $X$ is better than the answer to $Y$. Rank $X$ ahead of $Y$ when $p$ exceeds $1/2$ and rank $X$ behind $Y$ when $p$ is less than $1/2$. If $p$ equals $1/2$, declare a tie between $X$ and $Y$. (By virtue of our tie-resolution procedure, $p(X,Y) + p(Y,X) = 1$, implying the ranking does not depend on the sequence in which we draw the two answers.)
The calculation is a simple exercise for "just" a programmer (and a fun one if you are interested in efficient calculation, although that's unlikely to matter here). To make this proposal clear, though, I will illustrate it. Suppose all answers are on an integral scale from one to four, with four best. Write the answer distributions in the form $(k_1, k_2, k_3, k_4)$ where $k_3$ counts the number of "3"'s among the answers to a question, for example. For this example suppose $X$ has distribution $(4, 2, 0, 4)$ and $Y$ has distribution $(1, 6, 1, 2)$ (ten answers each). (Stop for a moment to consider which of these distributions ought to be considered "best" and note that they have identical means of 2.4 and identical medians of 2, suggesting this is a difficult comparison to make.) Then:
- There is a 4/10 chance of drawing a "4" for $X$. In this case,
- There is a 2/10 chance of drawing a "4" for $Y$ for a tie;
- There is an 8/10 chance of drawing less than "4" for $Y$, a win for $X$.
This contributes $(4/10)[(2/10)0.5 + 8/10] = 0.36$ to $p(X,Y)$. Continuing similarly,
- Drawing a "3" for $X$ is impossible; it contributes $0$ to $p(X,Y)$.
- Drawing a "2" for $X$ contributes $(2/10)[(6/10)0.5 + 1/10] = .08$.
- Drawing a "1" for $X$ contributes $(4/10)[(1/10)0.5] = 0.02$.
Whence $p(X,Y) = 0.36 + 0.00 + 0.08 + 0.02 = 0.46$. Because this value is less than $1/2$, we conclude $X$ should be ranked lower than $Y$.
This idea is related to that of Pitman Closeness and to certain non-parametric slippage tests (which decide whether one distribution has "slipped"--changed values--with respect to other distributions based on random samples of them), such as the Mann-Whitney (aka Wilcoxon) test.