1
$\begingroup$

I have data of the following form:

Rating 1 2 3
control 0 20 11
treatment 6 14 12

Where 1 is a plant of top quality, 2 is a plant of lesser quality that USED TO BE top quality, and 3 is a plant of poor quality that USED TO BE of type 2 quality. The values listed are the counts of each type of plant from an untreated control group and with a treated group.

I had been using a simple chi-square test to determine whether these treatments had any effect, but I've come to learn that that test assumes no order to the categories, whereas my categories do have an order.

Can someone please help me to understand how to determine:

A) whether the control and treatment are statistically significantly different from one another while taking into account the ordering of the categories.

B) How to determine an effect size from these results to use in statistical power calculations to determine sample size requirements for future studies.

For example, ordinal regression has been suggested, but it's not clear to me how such a calculation would be performed in this case (What is the dependent variable? How does one determine the significance parameter? How is effect size determined?)

Another suggestion has been the Kruskal-Wallis test, but I'm not clear how the order of the values is represented in that test.

Thanks for any advice that you can provide.

Improve this question
$\endgroup$
1
  • $\begingroup$ Mann-Whitney Wilcoxon rank sum test for zeroth-order stochastic dominance of treatment vs controls? $\endgroup$
    – Alexis
    Mar 18 at 6:12

1 Answer 1

Reset to default
5
$\begingroup$

I'll assume you want to treat Treatment as the independent variable and Rating as the dependent variable.

You can use the Wilcoxon-Mann-Whitney test. Equivalently, you could use Kruskal-Wallis.

In most software applications, you would first need to convert your counts to "long format" data. Simply, your count of 6 for "treatment", "1" would be converted to six observations, each with Treatment="treatment" and Rating="1".

An effect size statistic would report the probability that the value for an observation in one Treatment group would be greater than an observation in the other group. Vargha and Delaney's A reports this. Similar measures that are scaled to a range of 0 to 1 are Glass rank biserial coefficient and Cliff’s delta.

There are alternative tests. Kendall correlation (particularly tau-c), Spearman correlation, and Cochran-Armitage test are all candidates.

Addition:

In response to some some of the comments, I added an analysis and results for this data in R using Kruskal-Wallis test and ordinal regression.

Control   = c(rep(1, 0), rep(2, 20), rep(3, 11))
Treatment = c(rep(1, 6), rep(2, 14), rep(3, 12))

Data = data.frame(Group = c(rep("Control", length(Control)),
                            rep("Treatment", length(Treatment))),
                  Rating = c(Control, Treatment))

######################

kruskal.test(Rating ~ Group, data=Data)

   ### Kruskal-Wallis rank sum test
   ###
   ### Kruskal-Wallis chi-squared = 0.59551, df = 1, p-value = 0.4403

######################

Data$Rating.f = factor(Data$Rating)

library(ordinal)

model = clm(Rating.f ~ Group, data=Data)

anova(model, type="II")

   ### Type II Analysis of Deviance Table with Wald chi-square tests
   ###
   ###       Df  Chisq Pr(>Chisq)
   ### Group  1 0.6063     0.4362
Improve this answer
$\endgroup$
9
  • $\begingroup$ Kruskal-Wallis makes sense if things are individually ranked (no duplicates), but is it really correct if there are repeated rankings? It seems that a set that is all ranked 2 would have the same rank some as another group that is half 1 and half 3 and that they would show no difference under this test even though they are obviously different. $\endgroup$
    – Wilhelm
    Mar 20 at 14:51
  • $\begingroup$ Yes, Kruskal-Wallis will handle ties fine in most software implementations. And, yes, (2,2,2,2,2,2) and (1,1,1,3,3,3) have the same average ranking, and so be equivalent responses according to Kruskal-Wallis. If you want these two to be different, you might consider a chi-square test of association. $\endgroup$
    – Sal Mangiafico
    Mar 20 at 15:35
  • $\begingroup$ Sal Magniafico thank you so much for your answer. Can you let me know what you mean by "most implementations? My plan was to use python, but the test seems simple enough to write up in Excel if I wanted to. Can you let me know what it means to "handle ties fine" and what it means to not do so? $\endgroup$
    – Wilhelm
    Mar 21 at 10:55
  • $\begingroup$ What I mean by "most implementations" is the relevant functions in, say, R and Python and SAS. ... It's a good idea to conduct the test by hand (Excel) and then you can always compare the test to the output from a software package. Be aware, tho, at least in the two-sample Wilcoxon-Mann-Whitney test (the two-sample case for Kruskal-Wallis) , software packages often have different options that will produce somewhat different output. ... If you do it by hand, you just need to follow the procedure that accounts for ties. (cont') $\endgroup$
    – Sal Mangiafico
    Mar 21 at 14:10
  • $\begingroup$ And that's what I mean by "handles fine". It follows such a procedure. And also, that the results are sensible. Like if you compare the results to ordinal regression, having many ties in Kruskal-Wallis gives similar results. $\endgroup$
    – Sal Mangiafico
    Mar 21 at 14:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.