# Reporting insignificant results using mean ranks

I have a number of non-significant results from a Kruskal-Wallis test and wondering the best way to report them.

My independent variable is 3 categories:

Group A: Uses product A; Group B: Uses product B; Group C: Does not use either product.

The dependent variables are ordinal/Likert agreement scale.

Would it make sense to say while no statistical difference was found between Group A, Group B, and Group C, in many cases, Group A and Group B had higher mean ranks than Group C? For example, could I say something like: "A comparison of mean rank scores demonstrates that one of the two types of product users had higher mean rank scores than non-users on the following statements (S1, S2, S3...)?

• What message are you trying to convey? That you think the reader should attach any meaning to differences you have declared are not significant? – whuber Feb 27 at 19:58
• Thank you for posing the question. I can see how providing this information wouldn't be particularly useful to readers. Many thanks! – user3424836 Feb 27 at 20:02
• You might want to look at an effect size statistic that will be easy to interpret by your audience. Vargha and Delaney's A reports the probability of an observation in group 2 being greater than an observation in group 1. – Sal Mangiafico Feb 28 at 0:39
• Thank you! I found your explanation of Vargha and Delaney's A on your excellent website. I am using Stata, so wondering if is there any way to run this test on Stata? I'm also still wondering: if I report the effect size for non-significant results, is that information even useful to include? Many thanks! – user3424836 Feb 28 at 15:29
• VDA is used for the two-sample case, which I think would make sense for what you have asked. VDA is easy to calculate from the U or W statistic from a Mann-Whitney or Wilcoxon test. Instructions are in this document. P = U/(n1 * n2) and U = W - n.smaller * (n.smaller +1) / 2. If your data aren't too unwieldy, you could also run the following R code on this site. A = c(1,2,3,4,5,6,7,8,10); B = c(5,6,7,8,9,10); library(effsize); VD.A(B, A) – Sal Mangiafico Mar 1 at 14:40

One approach might be to report an effect size statistic. Vargha and Delaney's A reports the probability of an observation in one group being greater than an observation in the other group. I think this will be relatively easy for your audience to interpret. For more rigor, a confidence interval for the statistic can be reported. VDA is used for two groups.

VDA is easy to calculate from the U or W statistic from a Mann-Whitney or Wilcoxon test. Instructions are in this document.

P = U/(n1 * n2)
U = W - n.smaller * (n.smaller +1) / 2.


(Note: Oddly, it appears that the "W" that R reports is the same as U.)

If your data aren't too unwieldy, you could also run the following R code on this site or in R.

if(!require(effsize)){install.packages("effsize")}
A = c(1,2,3,4,5,6,7,8,10)
B = c(5,6,7,8,9,10)
library(effsize)
VD.A(B, A)

### Vargha and Delaney A
###
### A estimate: 0.75 (large)


Note that the effsize package will return a confidence interval for Cliff's delta, which is linearly related to VDA. Package documentation is here.

cliff.delta(B, A)

### Cliff's Delta
###
### delta estimate: 0.5 (large)
### 95 percent confidence interval:
###      lower     upper
### -0.1213262 0.8398131


VDA = Cliff's delta / 2 + 0.5

So, converting the output for Cliff's delta, VDA = 0.75 (0.439, 0.920)