# Statistical test for two categorical ordinal variables with four levels

There are two categorical variables with four levels, so ordinal variables. Specifically, they are responders and non-responders with heart function that is graded from mild, mild-moderate, moderate, severe. So each patient will either be a responder or non-responder and heart function would fit into one of four grades. I am looking to see if responders or non-responders are more likely to have better heart function. This is complicated by small n-values, which range from 0 - 5 per category. Thanks for any insight!

If I understand your explanation correctly, you may have 25 responders and 22 non-responders, possibly with different distributions of heart-function scores 1 through 4 between the two groups. Here are fake data sampled in R, to provide an example:

set.seed(131)

res = sample(1:4, 25, rep=T, p=c(1,2,3,4))
sort(res)
[1] 1 2 2 2 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4
t.r = tabulate(res)
[1]  1  3 10 11

non = sample(1:4, 22, rep=T, p=c(4,3,2,1))
sort(non)
[1] 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 3 3 3 4 4 4
t.n = tabulate(non);  t.n
[1] 12  4  3  3

TAB = rbind(t.r, t.n); TAB
[,1] [,2] [,3] [,4]
t.r    1    3   10   11
t.n   12    4    3    3


The resulting contingency table has some small observed counts, which result is less-than-ideal expected counts in a chi-squared test.

chisq.test(TAB)

Pearson's Chi-squared test

data:  TAB
X-squared = 17.672, df = 3, p-value = 0.000514

Warning message:
In chisq.test(TAB) :
Chi-squared approximation may be incorrect


Expected counts below $$5$$ may lead to incorrect P-values. (Many statisticians would say that a few expected counts of at least 3 amongst counts predominantly above 5 would be OK.)

chisq.test(TAB)\$exp
[,1]     [,2]     [,3]     [,4]
t.r 6.914894 3.723404 6.914894 7.446809
t.n 6.085106 3.276596 6.085106 6.553191


However in R, using chisq.test with parameter sim=T, it is often possible to simulate a reliable P-value for this test, even when some expected counts are too low. For our fake data, the P-value is sufficiently small to reject the null hypothesis that responders and non-responders have the same distributions of scores.

chisq.test(TAB, sim=T)

Pearson's Chi-squared test
with simulated p-value
(based on 2000 replicates)

data:  TAB
X-squared = 17.672, df = NA, p-value = 0.0004998


Ordinarily, I would not suggest a two-sample Wilcoxon rank sum test for two reasons: (a) many ties in the data; (b) markedly different shapes of distributions for responders and non-responders. However, the Wilcoxon RS test does find a significant difference.

wilcox.test(res, non)

Wilcoxon rank sum test
with continuity correction

data:  res and non
W = 448.5, p-value = 0.0001292
alternative hypothesis:
true location shift is not equal to 0
Warning message:
In wilcox.test.default(res, non) :
cannot compute exact p-value with ties


Moreover, empirical CDF (ECDF) plots of the two distributions show that the distribution of responders stochastically dominates (tends to have larger values than) the distribution of non-responders. The former distribution (blue) lies to the right (hence also below) the latter (brown).

There may be occasions where the Wilcoxon test or the ECDF plots may help to explain a difference in distributions found by the chi-squared test.

• Thank you for this thorough examination of my problem. My sample size is even lower than this. There are 5 responders and 7 non-responders. How would a Wilcoxon test be able to be used here? I thought the Wilcoxon test was for comparing means between non-parametric groups? Separately, I realize now I could assign each category of heart function a number. For example, assigning a number of 1 through 4 for ascending heart function. This would instead give a mean for each patient group and then allow for a non-parametric comparison test. Feb 1, 2021 at 14:17
• I'm not sure you have enough observations for a Wilcoxon test to be useful. Feb 1, 2021 at 18:22
• For binary Y the minimum sample size needed to only estimate the intercept is n=96. That's before you add regression coefficients. This situation is futile. Sep 22, 2023 at 11:34