I have 10 independent samples of ordinal data ranked 1-3. I also have predicted ranks for the same data set. I want to test if the predicted ranking matches the observed ranking at a rate better than chance. A similar example might be a set of 10 races among 3 contestants where each race has different people in the races and we want to see if we can predict the order they finish in better than chance. Our data might look like so:
| Race Number | Pred Place 1st | Pred Place 2nd | Pred Place 3rd |
|-------------|----------------|----------------|----------------|
| 1 | 2 | 1 | 3 |
| 2 | 1 | 3 | 2 |
| 3 | 3 | 1 | 2 |
| 4 | 3 | 2 | 1 |
| 5 | 3 | 2 | 1 |
| 6 | 3 | 1 | 2 |
| 7 | 1 | 2 | 3 |
| 8 | 1 | 2 | 3 |
| 9 | 1 | 3 | 2 |
| 10 | 1 | 2 | 3 |
Where Race Number
is an indicator for which race we are looking at Pred Place 1st
is the predicted place for the contestant who actually came in first in that race, Pred Place 2nd
for the one who came in second, etc.
I know for a single paired ordinal sample the Wilcoxon signed-rank test would be used, but I am not sure what to do with multiple samples. Would this be a good application of Fisher's method? Also with only 3 values to rank, my Wilcoxon test only returns p-values of 1 or NaN, is there another test that would be more appropriate here?