Rosenthal and colleagues (Rosenthal, Rosnow & Rubin, 2000) suggest that it is possible to compute r (correlation coefficient) as an effect size from repeated measures designs (e.g., by converting the t-statistic of a paired t-test). Assuming that I accept this approach (I know that this is always debatable), and I am also using Fisher's r-to-z conversion in a meta-analytic context, how would I compute the sampling variance for my transformed z? As the sampling variance is typically estimated by V = 1 / (n-3), I am wondering what "n"-value would be appropriate...would it be the number of "pairs", that is, for a study with n = 30 participants measured on a variable X in two experimental conditions (treatment A, treatment B), would it make more sense to work with n = 30 (i.e., the number of paired measures), or n = 60 (which essentially means we are treating the study as if it were an indepedent-group design). Any ideas?
-
$\begingroup$ It does seem rather indirect when you have the option to proceed with the means but no doubt you have your reasons. It is hard to believe that the answer can be 60. $\endgroup$– mdeweyMar 4, 2016 at 16:57
-
$\begingroup$ How do Rosenthal and colleagues propose to convert the t-statistic to r? (i.e., could you add the equation to your question?). $\endgroup$– WolfgangMar 5, 2016 at 15:52
-
1$\begingroup$ @KristiLo That equation is appropriate for converting an independent samples t-test into a point biserial correlation. It is not applicable to the paired samples t-test. Also, Fisher's r-to-z transformation is for Pearson product-moment correlations, not point biserial correlations, so the $1/(n-3)$ equation for the variance does not apply, regardless of what value for $n$ you plug into it. $\endgroup$– WolfgangMar 8, 2016 at 8:55
-
1$\begingroup$ @KristiLo What would be the interpretation of the value obtained in this manner? It's not a Pearson product-moment correlation between two continuous variables. It's not a point-biserial correlation between a dichotomous variables (pre vs post) and the continuous variable. Unless the value is interpretable, there is no point even wondering about its variance (before or after applying a transformation to it that is only meant to be used for Pearson product-moment correlations). $\endgroup$– WolfgangMar 8, 2016 at 19:34
-
1$\begingroup$ why do you want to do your analysis using r when it looks like you could use the mean difference and its standard error? As @Wolfgang has suggested you seem to have got bogged down in the many meanings and uses of r and gone up a blind alley. $\endgroup$– mdeweyMar 8, 2016 at 22:02
1 Answer
So this does not go unanswered here is content from the comments mostly by the OP and by @Wolfgang.
The OP clarified that the formula she mentioned was:
he formula given is as follows: sqrt $(t^2$ / $(t ^2 + df))$. The problem is, however, that I only have descriptives (means and SDs) for the two paired conditions, not the actual t-statistic. I thought I could use the descriptives to calculate d and convert this to r for comparative purposes
To which Wolfgang replied:
That equation is appropriate for converting an independent samples t-test into a point biserial correlation. It is not applicable to the paired samples t-test. Also, Fisher's r-to-z transformation is for Pearson product-moment correlations, not point biserial correlations, so the 1/(n−3) equation for the variance does not apply, regardless of what value for n you plug into it
In answer to a comment by me the OP replied:
I am running a meta-analysis on correlations, as most studies I am considering actually measure the variables of interest on a continuous scale. Some of the studies manipulate one of the variables (e.g., degree of "learner control") experimentally, sometimes using between-subjects designs, but in other cases, using within-subjects designs. I wanted to include these different designs (I know this is debatable) in the same meta-analytic model and look at design as a potential moderator. But doing so, in the end, would require me to have the same ES statistic throughout the whole set of studies
And as my parting shot:
I think you have two main options now. Option 1 - Meta-analyse each study design separately giving you (I think) three analyses and do a narrative comparison. Option 2- do a meta-analysis of the significance levels (p-values) overall. Option 1 gives you the advantage of effect sizes but only in three separate syntheses, option 2 enables all studies to be taken together but you lose the effect sizes.
