How to perform a meta-analysis on change from baseline after a treatment?

Question

I am trying to run a meta-analysis calculating the mean effect size of a manual therapy treatment on pressure pain threshold pre-post treatment (i.e. within-group change), so there is no comparison to sham/control. I will be using mean change from baseline (continuous) and I have SDs/CIs of the change. There are 10 studies, and most studies have multiple testing sites (e.g. at the lower back and at the calf) so each site needs to be included separately. I’ll need to run it under a random-effects model. Can I do this? And if so, how do I go about it?

Most information out there about meta-analysis is very focused on inter-group comparisons so I've been unable to find a clear answer.

If I can, I’m uncertain of which programs would allow me to do it. Presumably I could use R since there are so many packages available – any suggestions on packages that might suit? Based on a trial of the program Comprehensive Meta-Analysis it seems to do what I’m looking for, but is rather expensive ($500 for 1 year license, unless I purchase with my own money as a student). I have also looked into RevMan, but from what I can gather it only supports inter-group comparisons, since Cochrane reviews are typically looking at mean differences comparing a treatment against control/placebo/gold standard. Please correct me if I am wrong here. Are there any other good programs that are cheap/free that would do what I want?

Any help is appreciated, thanks.

Answer 1

There is no need to spend money. There are several options available in R of which both metafor and meta are actively maintained and developed. There are also Stata solutions if you have access to it.

The meta analysis model just needs an effect size and its standard error from each study. It neither knows nor cares whether this comes from a comparative study or a non-comparative study. I would start by analysing each site separately although the canonical analysis would use all in asingle model.

Just a final comment, R is only free if you do not charge for the time you spend learning it.

Answer 2

The pre-post design is so weak that I'm not sure it's worth spending a lot of time on this. Among other things it suffers from regression to the mean and Hawthorne effects, not to mention general uncontrolled time trends.

Answer 3

There is a reason why there is less information on the meta-analysis of within-group changes. They are not a useful thing to meta-analyze, if you wish to say something on the efficacy or effectiveness of therapies. Particularly, if you have an inclusion criterion for a study or analysis that requires a certain pre-treatment level, then you are more or less guaranteed an improvement (very often by substantial amounts) simply by regression to the mean (never mind the Hawthorne effect).

The only cases where it makes some sense, is if you informally or formally compare to some known natural history for patients that have been selected in a similar manner (incl. inclusion criteria) and there is quite a literature on such indirect comparisons. To summarize: unless you have a massive effect that you are studying, it is typically very, very hard to reliably conclude whether an intervention compared to historical controls does something.

In principle, the calculations you would want to do, if change from baseline were a meaningful thing to meta-analyze would be quite straightforward to implement using e.g. R (many packages such as metafor, brms etc. would be suitable). And, yes, I would think that one should account for non-independent outcomes (e.g. different testing sites reported in the same study) using a random effect.

Answer 4

I agree with comments and answers that discuss the weaknesses of examining treatment effectiveness based solely on within-group change. Setting those issues aside and focusing simply on how to do this, it is rather simple. For each study, you need to compute a standardized mean change or gain score. You also need to compute the variance of this effect size. The effect size is computed as: $$ d = \frac{\overline{X}_{2} - \overline{X}_{1}}{\sqrt{\frac{s^2_{2} +s^2_{1}}{2}}} $$ The variance, needed for the inverse variance weight, is $$ v_d = \frac{2(1-r)}{n} + \frac{d^2}{2(n)}$$ The complication is that $r$ is the pre-post correlation. This is rarely provided by the authors. However, it can be estimated from a paired t-test, assuming you also have the means and standard deviations. $$ r = \frac {\left(s_1^2 t^2 + s_2^2 t^2 \right) - \left(\overline{X}_2 - \overline{X}_1\right) n} {2 s_1 s_2 t^2} $$ You can then analyze thiis as any meta-analysis using Stata, R, or SPSS (for macros for SPSS and Stata, see my website: http://mason.gmu.edu/~dwilsonb/home.html).