Cohen's d, but using mean absolute deviation

Question

A common measure of effect size used in the social sciences is to look at the standardized difference in means between two conditions (Cohen's d). That is, take the difference in means between two independent samples, and divide that number by their pooled standard deviation.

Here's my question. I wonder if you can calculate a similar statistic, but instead of using the pooled standard deviation, to use the pooled mean absolute deviation? There is some empirical work suggesting that people often interpret standard deviations as mean absolute deviations [1]. Plus, it strikes me that an effect size normalized over mean absolute deviations would be more intuitive/easier to understand than Cohen' d. Instead of thinking of effect size in terms of standardized units, you would instead think in terms of differences normalized over the mean spread.

So, I'm wondering if there are good mathematical reasons why one would want to use SDs instead mean absolute deviations? Please forgive me if this is a sophomoric question, my mathematical intuitions are fairly limited.

[1] Goldstein, D. G. & Taleb, N. N. (2007). We don't quite know what we are talking about when we talk about volatility. Journal of Portfolio Management, 33(4), 84-86.

Answer 1

One problem with "pooling" mean deviation* rather than standard deviation is that mean deviations "don't add", while variances do.

That is, for independent random variables $A$ and $B$, $\text{Var}(A+B)= \text{Var}(A)+\text{Var}(B)$ and since $\text{Var}(-B)= \text{Var}(B)$, we thereby also get $\text{Var}(A-B)= \text{Var}(A)+\text{Var}(B)$. This works for any pair of random variables (given that independence).

A closely related result to do with partitioning sums of squares works in samples (or you can cast it in terms of n-denominator variances of samples by treating the ecdf as the population cdf).

From these facts (plus the additional basic variance fact that $\text{Var}(kA)=k^{2}\text{Var}(A)$) we can get to a simple form for a "pooled" estimate for the variance of $\bar{X}-\bar{Y}$.

Since this always works (samples or populations, normal or non-normal), we can then take advantage of the large sample properties of the sample mean and so forth and produce statistics that "make sense" even if the assumptions (the things you rely on under the null to derive the t-distribution for the usual two sample t-test) don't quite hold exactly $-$ which in practice they never quite do.

The same cannot be said for mean deviation.

It is not the case that $\text{MD}(A+B) = \text{MD}(A)+\text{MD}(B)$ in general. Nor do we have $\text{MD}(A+B) = \sqrt{\text{MD}(A)^2+\text{MD}(B)^2}$ in general.

Indeed there's no corresponding sample result either.

So in general we cannot proceed from the sample mean deviations $\text{MD}(x)$ and $\text{MD}(y)$ to some $\text{MD}(\bar{x}-\bar{y})$.

While you could potentially construct an estimator from $\text{MD}(x)$ and $\text{MD}(y)$ that would under specific sets of assumptions act as a reasonable estimator for the population quantity $\text{MD}(\bar{X}-\bar{Y})$, if you change the assumptions, it would generally lead to a different estimator.

You could certainly do something with a bootstrap test, and perhaps even with a permutation test, but you would not necessarily end up with a nice set of simple large-n statistics that work fairly broadly as now. You might end up with some specific approximations, different in different situations, and a potentially substantial set of caveats about when they don't really work very well or lead to counterintuitive results.

That neat property of variances (and hence, simple ways of combining standard errors in sums and differences of averages) is why it "sticks around". It's not just technological inertia (i.e. it's not just "we learn it because that's the way we've always done it").

Things do actually get more complicated when you move away from standard errors.

Teachers of mathematically weak students probably don't want to have to replace the simple things they teach now with teaching a variety of computational tools like bootstrapping in a beginners class -- albeit that may potentially carry some advantages as well as the numerous difficulties.

It's not necessarily an insurmoutable task by any means but it's also not as simple as "just replace $s$ by $MD$ and you're done".

Note that we haven't got as far as constructing a corresponding effect size estimate to go with such a test; even the first step (getting a test it relates to) is not quite as obvious a thing as it might seem.

---

*(traditionally the "... absolute deviation from the mean" part used to be taken as read if not otherwise specified; I will continue to do so)