Is it correct to model percent change as a continuous variable?

Question

I have data from a longitudinal study where there are 46 subjects, each with 13 observations over time on a given outcome measure. A prior analysis conducted on this data performed a linear mixed model on the percent change (treatment, baseline value, time, and treatment*time were independent variables in the model). I am wondering if using raw change as the outcome variable is more correct, especially since baseline value is controlled in the model anyway.

Answer 1

The best way to do this is to model the final score as a function your listed independent variables PLUS the baseline score, like this:

final ~ baseline + treatment + time + treatment*time

There are a few important considerations:

1. You should not model % change (final-baseline/baseline) and include baseline as a predictor, because these variables are structurally correlated. Even worse might be to model raw change (final - baseline) with baseline as a predictor. As baseline changes, you expect the dependent variables to change in both cases, all else equal. Conclusion: If you use % or raw change, do not include baseline as a predictor.

2. Modelling percentages as continuous variables is a dying approach, because these variables are typically not normally distributed, by virtue of being bounded at 0 and 1, and also see last comment by TiffTiff. The arcsine square-root (aka angular) transformation has traditionally been used to combat this problem, but is also falling out of favor due to highly conditional efficacy (i.e., it often doesn't work well).

3. Modelling raw change, even when baseline is not included as a predictor, is not ideal because you in essence constrain the relationship between baseline and final to be 1:1. Basic algebra to see why:

Here is the raw change model:

final - baseline ~ treatment

which is more precisely,

1*final - 1*baseline ~ b0 + b1*treatment + error

where b0 and b1 are parameters to be estimated. if you rearrange, you get:

1*final ~ b0 + 1*baseline + b1*treatment + error

hence, the parameter describing the relationship between baseline and final is set to 1.0.

If instead you model like this:

final ~ baseline + treatment

that is more precisely

1*final ~ b0 + b1*baseline + b2*treatment + error

If indeed there is not a 1:1 relationship between baseline and final, the parameter estimate b1 will be greater than or less than 1. If it turns out that b1 ≠ 1, you will have more power to test the effect of treatment than if you used raw change, and also get information about the relationship between baseline and final. On the other hand, if b1 = 1, then you should have less power, since you're using up one more degree of freedom.

As far as reporting these stats, I understand why you want to say "The difference between scores increased when the treatment was applied": You want to control for the baseline. But that's exactly what the final ~ baseline + treatment model does, and it does it better. You can say "Scores were higher with the treatment, controlling for the baseline score".

Answer 2

Whether this is the best approach depends on what your data looks like, and why the percent change was used. For example, % change is fairly easy to understand by laypeople, and so it's sometimes preferred when that's the target audience. Including the baseline as a covariate somewhat complicates how that percent would be stated, though, since the expected change would depend on the initial value.

Certainly, to use percent change, the data must be on a scale where 0 means 0. That is, a 0 must mean an absence of disease or similar.

I think this article does a pretty good job of summarizing some problems associated with modeling percent change: http://allenfleishmanbiostatistics.com/Articles/2012/06/18-percentage-change-from-baseline-great-or-poor/.

In all cases, Analysis of Covariance (ANCOVA) with baseline as the covariate was the most efficient statistical methodology. Analyzing the change from baseline “has acceptable power when correlations between baseline and post-treatment scores are high; when correlations are low, POST [i.e., analyzing only the post-score and ignoring baseline – AIF] has reasonable power. FRACTION [i.e., percentage change from baseline – AIF] has the poorest statistical efficiency at all correlations.”

[Note: In ANCOVA, one can analyze either the change from baseline or the post treatment scores as the d.v. ‘Change’ or ‘Post’ will give IDENTICAL p-values when baseline is a covariate in ANCOVA.]

As an example of his results, when the correlation between baseline and post was low (i.e., 0.20) the percentage change was able to be statistically significant only 45% of the time. Next worse, was change from baseline with 51% significant results. Near the top was analyzing only the post score at 70% significant results. The best was ANCOVA with 72% significant results.

Furthermore, percentage change from baseline “is sensitive in the characteristics of the baseline distribution.” When the baseline has relatively large variability, he observed that “power falls.”

Also a potential concern: the ratio will blow up as the denominator approaches 0, which can create obvious problems when modeling.