I have a table that looks like something this:
Outcome
Method 3 2 1 0
1 x_11
2 x_21 ....
3 x_31
4 x_41 x_44
Each x_ij represents a count. Also for each method, there were 140 tests. Thus \sum_j (x_ij) = 140.
I want to see which method is best. Outcome 3 is better than outcome 2 which is better than outcome 1 etc..
The first thing I thought to do was a chi-square test of independence. That will tell me if there is an association between outcome and method.
Of course, I can, for each method collapse the outcome into two groups (say group 1 consists of just outcome 3 and group 2 consists of outcomes 2-0) and then rank the percentages and test to see if they are different.
However, what I'm really looking for is to somehow figure out which Method is best.
One thought I had was to multiple outcome * count for each method to come up with a weighted average. For instance, for method 1, I would multiple x_11 *3 + x_12 *2 +...+x_14*0. However, how would I compare those values? Using some sort of permutation test?
Thanks!
