When calculating a standardized change score, what do you divide by and why?

Question

I understand that, in most cases, one calculates a standardized change between two scores as

(Lakens 2013)

i.e., the mean of the change scores is divided by the standard deviation of the change scores.

My question is: Why don't we divide by the standard deviation of the scores at baseline, instead? It seems more intuitive to me to present a standardized change as a measure of variability in the scores at baseline rather than variability in how much scores changed.

For example, imagine a medication for headaches; at baseline, people's headache pain scores have a mean of 10 and a standard deviation of 2. After taking the medication, their headache pain scores have a mean of 8. The drug works about the same in everybody; the standard deviation of the change score is only .25. Wouldn't it be more useful to say "this drug helped move the average person a whole standard deviation down from their baseline pain score" rather than "this drug moved the average person down eight times what the average person changed"?

What is the rationale here?

Answer 1

A Cohen's $d$ is a ratio: the observed effect divided by a standard deviation (SD). The SD is a "unit of measurement" and you are correct that there are different choices for the SD in the denominator of a Cohen's $d$.

For example, the (Lakens, 2013) paper introduces three Cohen's $d$ effect sizes for within-subject comparisons (and generally recommends to report $d_{rm}$ or $d_{av}$, not $d_z$).

$$ \begin{aligned} d_z &= \frac{M_\text{diff}}{\sqrt{\operatorname{SD}_\text{pre}^2 + \operatorname{SD}_\text{post}^2 - 2r\operatorname{SD}_\text{pre}\operatorname{SD}_\text{post}}} \\ d_{rm} &= \frac{M_\text{diff}}{\sqrt{\operatorname{SD}_\text{pre}^2 + \operatorname{SD}_\text{post}^2 - 2r\operatorname{SD}_\text{pre}\operatorname{SD}_\text{post}}}\sqrt{2(1-r)} \\ d_{av} &= \frac{M_\text{diff}}{\left(\operatorname{SD}_\text{pre} + \operatorname{SD}_\text{post}\right) / 2} \end{aligned} $$ where $\operatorname{SD}_\text{pre}$ is the standard deviation at baseline (before treatment), $\operatorname{SD}_\text{post}$ is the standard deviation after treatment and $r$ is the correlation between the measures.

Your argue in favor of another variant:

$$ \begin{aligned} d_{?} &= \frac{M_\text{diff}}{\operatorname{SD}_\text{pre}} \end{aligned} $$

Note that if the treatment doesn't change the standard deviation, then $d_{?} \approx d_{av}$. However, it might be the case that the treatment is very effective for some patients while not effective at all for others. That is, the $\operatorname{SD}_\text{post} > \operatorname{SD}_\text{pre}$. The Cohen's $d$ you propose won't be sensitive to the increase in variability and $d_{?} > d_{av}$. (Also consider the case where the variance decreases post-treatment.) The question is: If that were the case, do you think $d_{?}$ or $d_{av}$ is a better effect size measure?

[1] D. Lakens. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in psychology, 4:863, 2013.