1
$\begingroup$

I am aware that you must be careful about when you can compare effect sizes. But let's assume that I have two effect sizes from two similar studies: similar groups, same intervention, same measurement. Let's say, two classes in the same school receiving the same intervention and measured on the same outcome. If one class produces an effect size (Cohen's d) of 0.4 and the other produces an effect of 0.2, is it reasonable to describe that the former has an effect that is twice the size of the latter [scale data]? Or can I only say that the former is larger than the latter, without specifying a factor [ordinal data]?

$\endgroup$
2
$\begingroup$

In Stevens's level-of-measurement framework, being able to say “twice as much” is associated with the ratio level of measurement and implies a meaningful and unique 0. If the origin of the scale is arbitrary, the measure is not automatically merely “ordinal” but could also be “interval-level”.

Intuitively, it seems to me that Cohen's d is obviously not ordinal, it provides more information than that. I also think that “twice as much” could make sense for a (standardized) difference but I guess that whether or not it is a useful description could depend on the meaning of the original measures and not only on the description being “allowed” in Stevens's framework.

PS: Note that while it is somewhat tangential to your specific question, the usefulness of this framework and of notions like “ordinal measurement” to guide statistical analysis is in fact disputed.

$\endgroup$
  • $\begingroup$ Thanks very much for your response. You're right: I was thinking of ratio when I said scale. This is not for detailed statistical work, but I was interested in the intuition of whether one effect size could be thought to be a scale factor of another comparable effect size. $\endgroup$ – Neil Brown Aug 4 '13 at 11:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.