Effect size calculation for comparison between medians I have two groups of samples - disease and normal. I have calculated whether there exists any statistically significant difference between the medians of the two groups via the Mood's median test. The p-value results indicated that I have a significant difference with an $\alpha$ of 0.1. Now I want to calculate the effect size between the two groups. I was planning on using Cohen's d, which is given by $d=m_1-m_2/s_p$, where $m_1$ and $m_2$ are the means of the two groups and $s_p$ is the pooled standard deviations for the two groups. My question is whether I can modify this formula to calculate the effect size for the medians between two groups,where $m_1$,$m_2$ will be medians instead of means. Is it the right way to do that?
 A: Cohen's d is probably not the best effect size statistic for a comparison of medians, since it is based on the mean and standard deviation(s), and is related to the calculations used in a t test.
I haven't seen this anywhere that I remember, but I suppose you could use an analogous statistic that uses the differences in medians and the median absolute deviation.
(median(B) - median(A)) / sqrt((mad(A)^2 + mad(B)^2)/2)

For normal samples, this will give a similar result to Cohen's d.
If I invented this, you can call it Mangiafico's d. :)
Of course, a simple difference in medians is also a useful effect size statistic.  It isn't unit-less, but has units of the measurements.  If you add confidence intervals, this is quite informative.
Finally, effect size statistics that might be used for a Wilcoxon-Mann-Whitney test would be useful:  Vargha and Delaney’s A, Cliff’s delta, and the Glass rank biserial coefficient.
Addendum November 2022:
I added an implementation of the statistic that uses the differences in medians and the median absolute deviation to my rcompanion package.  An example follows.
Group1 = c(0,  4,  8, 12, 16, 1,  5,  9, 13, 17,  2,  6, 10, 14, 18)
Group2 = c(8, 12, 16, 20, 24, 9, 13, 17, 21, 25, 10, 14, 18, 22, 26)

library(rcompanion)

mangiaficoD(x=Group1, y=Group2, verbose=TRUE, ci=TRUE, hist=TRUE)

   ###   Group  Statistic Value
   ### 1     A     Median  9.00
   ### 2     B     Median 17.00
   ### 3       Difference -8.00
   ### 4     A        MAD  7.41
   ### 5     B        MAD  7.41
   ### 6       Pooled MAD  7.41
   
   ###       d lower.ci upper.ci
   ### 1 -1.08    -2.81   -0.169

Group = factor(c(rep("I", length(Group1)), rep("II", length(Group2))))
Value = c(Group1, Group2)
Data = data.frame(Group, Value)

plot(Value ~ Group, data=Data)

library(coin)

median_test(Value ~ Group, data = Data)

    ### Asymptotic Two-Sample Brown-Mood Median Test

    ### Z = -2.1589, p-value = 0.03086

Addendum 2
I mentioned Vargha and Delaney’s A above as an option for effect size.  This statistic is based on the proportion of observations in Group A that are greater than observations in Group B, and so on.
For the example above, the proportion of observations in Group1 > Group2 is 0.16, and  the proportion of observations in Group2 > Group1 is 0.80.
These statistics are useful to report when comparing two medians.
library(rcompanion)

vda(x=Group1, y=Group2, verbose=TRUE)

   ###            Statistic Value
   ### 1 Proportion Ya > Yb  0.16
   ### 2 Proportion Ya < Yb  0.80
   ### 3    Proportion ties  0.04
   
   ###  VDA 
   ### 0.18 

A: Mood's test is really just a chi-square test in disguise (https://en.wikipedia.org/wiki/Median_test).
Hence, I recommend using the measure of effect size for chi-square tests. There are three standards used: phi, the odds ratio, and Cramer's V. The first two are only defined when comparing two samples, while Cramer's V works for nxn contingency tables.
Here are equations for phi and V:
$$\phi= \sqrt(\chi^2/n)$$
$$V=\sqrt(\chi^2/(n \min(c-1, r-1))$$
where n is the total number of observations, $\chi^2$ is the test statistic, and r/c are the dimensions of the contingency matrix (in this case it will be two x the number of groups you are comparing). If you are interested in the odds ratio, I would recommend googling it, as it is a little bit more complicated than the others.
Note I leave interpretation of the effect size to the reader. :)
https://www.real-statistics.com/chi-square-and-f-distributions/effect-size-chi-square/
