0
$\begingroup$

I would like to perform something like two-sample Wilcoxon tests/T-test (paired = FALSE) to compare two method's expected scores.

Here is the table of counts:

table(data$method_type, data$score)

   Score:  3  4  5  6
  method1  2  3 12 13
  method2  1  6 15  8

Experiment was designed, such that score could be from set {0,1,2,3,4,5,6}, which represents points from exam test (0 point worst, 6 point best)

I have another dataset, where score (let's named it score_2), where:

table(data$method_type, data$score_2)

  Score_2  0  1  2  3  4  5  6
  method1  1  2  1  5  5  3 13
  method2  1  0  4  7  5  3 10

Problem: there is just 4 unique values of score (3,4,5 and 6). I definitely can't use t-test, but what about wilcoxon (Mann-Whitney)? Can I use it? Or which test would you advice me?

Dataset:

   method_type score
1      method1     5
2      method1     6
3      method1     5
4      method1     4
5      method1     6
6      method1     5
7      method1     6
8      method1     3
9      method1     5
10     method1     6
11     method1     5
12     method1     5
13     method1     6
14     method1     4
15     method1     6
16     method1     5
17     method1     5
18     method1     6
19     method1     6
20     method1     5
21     method1     3
22     method1     6
23     method1     6
24     method1     5
25     method1     6
26     method1     5
27     method1     6
28     method1     5
29     method1     4
30     method1     6
31     method2     4
32     method2     5
33     method2     5
34     method2     6
35     method2     4
36     method2     6
37     method2     5
38     method2     5
39     method2     5
40     method2     3
41     method2     5
42     method2     6
43     method2     4
44     method2     5
45     method2     6
46     method2     5
47     method2     5
48     method2     6
49     method2     5
50     method2     4
51     method2     5
52     method2     6
53     method2     5
54     method2     5
55     method2     5
56     method2     6
57     method2     6
58     method2     5
59     method2     4
60     method2     4
$\endgroup$
  • 1
    $\begingroup$ You can consider gamma, Kendall’s tau-, Stuart’s tau-, and Somers’ D, and Spearman rank correlation coefficient. All of them consider the ordinal of the variables. Wilcoxon method you mentioned is not so good, because this method requires continue variable. $\endgroup$ – user158565 Dec 9 '18 at 3:36
1
$\begingroup$

Or which test would you advice me?

If your question is whether the score distribution with method 2 differs from the distribution with method 1 then you can use Fisher's exact test:

x <- matrix(
  c(1L, 2L, 1L, 5L, 5L, 3L, 13L,
    1L, 0L, 4L, 7L, 5L, 3L, 10L), 
  nrow = 2, byrow = TRUE, 
  dimnames = list(c("method1", "method2"), 0:6))

set.seed(18548777)
fisher.test(x, simulate.p.value = TRUE, B = 10000)
#R 
#R Fisher's Exact Test for Count Data with simulated p-value (based on 10000 replicates)
#R 
#R data:  x
#R p-value = 0.6969
#R alternative hypothesis: two.sided

The null hypothesis is that the rows and columns are independent. We cannot reject this. Another options is the Monte Carlo test implemented in chisq.test:

set.seed(18548777)
chisq.test(x, simulate.p.value = TRUE, B = 10000)
#R 
#R Pearson's Chi-squared test with simulated p-value (based on 10000 replicates)
#R 
#R data:  x
#R X-squared = 4.5246, df = NA, p-value = 0.6812

The conclusion is the same. We cannot reject that scores are independent of the of the method. The answer here though ignores that the data is ordinal.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.