1
$\begingroup$

In conducting a meta-analysis, I have pre-post data from single case designs. Unfortunately, I do not have access to the correlations between the pre and post scores. Is it correct that if I calculate the effect sizes using the independent measures, then the effect sizes will remain the same, but the standard error will increase and the confidence intervals will widen?

$\endgroup$
2
$\begingroup$

The formula for the standard error of the difference between two correlated means $m1$ and $m2$ is $$s_{md} = \sqrt{s^2_{m1} + s^2_{m2} - 2 r_{12} s_{m1} s_{m2}}$$

So if $r_{12}$ is positive the third term under the radical $2 r_{12} s_{m1} s_{m2}$ is also positive and so the standard error is reduced. By all means try a grid of values for the correlation but assuming it to be zero is conservative.

$\endgroup$
  • $\begingroup$ So, if we assume that the correlation between pre and post data is 0.5, I understand that d = Mean Difference/S Within. However, would it also require knowing the standard deviation of the difference scores? These are not listed in the articles. $\endgroup$ – Dan K Jun 13 '16 at 11:45
  • $\begingroup$ The formula shows you how to calculate them $\endgroup$ – mdewey Jun 13 '16 at 12:11
  • $\begingroup$ The effectsize has not been defined. what is effect size using indepenfent measures ? $\endgroup$ – Subhash C. Davar Jun 13 '16 at 14:10
  • $\begingroup$ Effect size here is being used in the sense it has in meta-analysis not in the sense of Cohen's $d$. So it could be anything: log odds ratio, log relative risk, log hazard ratio, proportion (possibly transformed), mean difference, standardised mean difference, correlation (presumably transformed), or anything else you can extract from the primary studies with a standard error. $\endgroup$ – mdewey Jun 13 '16 at 17:54
  • $\begingroup$ Good point and apologies for my ignorance; I'm learning as I go. I am calculating standardised mean differences and realised after I wrote the last comment that for Cohen's d I should divide the mean difference for the pre-post scores by the standard error formula that you provided, mdewey. Is that correct? I will have to assume the correlation, which could be anything from .3 to .7 I would imagine. That is quite a large margin though. $\endgroup$ – Dan K Jun 13 '16 at 21:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.