How to compare different effect sizes (d vs. r)?

Question

My question is whether I can compare two different effect sizes (Cohen's d and r). I have tested the differences between two paired groups regarding different variables. They are paired because they are measurements at two different time points. One parameter/variable isn't normally distributed and the other is normally distributed, so I did a Wilcoxon signed rank test and a paired t-test (each for different variables). Accordingly, I calculated different effect sizes, r and Cohen's d. What do I have to do to compare the effect sizes? In one parameter it is r = 0.58, in the other d = 0.78. Although d = 0.78 is a medium effect, the median of the second time point differs more clearly from the first time point than for the variable with r = 0.58 (which is a large effect).

Does anyone have any ideas?

Answer 1

If your main goal is to compare in-sample effect sizes between the two datasets, then it would probably make more sense to run them both through the same sort of model. On the one hand, people often diagnose too rigidly (or even wrongly) what sort of "normality" they need to run the parametric tests/models. On the other hand, there is little to lose from looking at both through a non-parametric lens (you might lose a little power on the p-value part, but not as much as people often think, and if your goal is to get two comparable effect sizes at the end of it, the inferential bit doesn't sound to be the thing you're most interested in). So, plenty of scope for running them both through the same sort of model.

The key point here is that parametric vs nonparametric is just a modelling decision -- and it's certainly not mutually exclusive. In many situations, you can find a justification for using both. While both tests pick at a similar effect size (i.e. how consistently do we see increases from one time-point to the other), they look from slightly different angles, and so will highlight or neglect different features in your data. So, while, they could probably be, superficially, converted into something the (on the surface) means the same thing, they will be looking from different angles at the same question.