Rosenthal and colleagues (Rosenthal, Rosnow & Rubin, 2000) suggest that it is possible to compute r (correlation coefficient) as an effect size from repeated measures designs (e.g., by converting the t-statistic of a paired t-test). Assuming that I accept this approach (I know that this is always debatable), and I am also using Fisher's r-to-z conversion in a meta-analytic context, how would I compute the sampling variance for my transformed z? As the sampling variance is typically estimated by V = 1 / (n-3), I am wondering what "n"-value would be appropriate...would it be the number of "pairs", that is, for a study with n = 30 participants measured on a variable X in two experimental conditions (treatment A, treatment B), would it make more sense to work with n = 30 (i.e., the number of paired measures), or n = 60 (which essentially means we are treating the study as if it were an indepedent-group design). Any ideas?
So this does not go unanswered here is content from the comments mostly by the OP and by @Wolfgang.
The OP clarified that the formula she mentioned was:
he formula given is as follows: sqrt $(t^2$ / $(t ^2 + df))$. The problem is, however, that I only have descriptives (means and SDs) for the two paired conditions, not the actual t-statistic. I thought I could use the descriptives to calculate d and convert this to r for comparative purposes
To which Wolfgang replied:
That equation is appropriate for converting an independent samples t-test into a point biserial correlation. It is not applicable to the paired samples t-test. Also, Fisher's r-to-z transformation is for Pearson product-moment correlations, not point biserial correlations, so the 1/(n−3) equation for the variance does not apply, regardless of what value for n you plug into it
In answer to a comment by me the OP replied:
I am running a meta-analysis on correlations, as most studies I am considering actually measure the variables of interest on a continuous scale. Some of the studies manipulate one of the variables (e.g., degree of "learner control") experimentally, sometimes using between-subjects designs, but in other cases, using within-subjects designs. I wanted to include these different designs (I know this is debatable) in the same meta-analytic model and look at design as a potential moderator. But doing so, in the end, would require me to have the same ES statistic throughout the whole set of studies
And as my parting shot:
I think you have two main options now. Option 1 - Meta-analyse each study design separately giving you (I think) three analyses and do a narrative comparison. Option 2- do a meta-analysis of the significance levels (p-values) overall. Option 1 gives you the advantage of effect sizes but only in three separate syntheses, option 2 enables all studies to be taken together but you lose the effect sizes.