I'm trying to do a meta analysis across a bunch of bird playback studies. Thanks to help on a previous post here I'm fairly certain I need to use inverse variance weighting (maybe with the 'meta' package in R).

These playback studies use different response measures with some overlap. Some studies use 10 response measures while others use one. I have the means and standard errors for each study's response measures. I want to look at if, across birds, these response measures increase along with another variable that I have for each bird.

I am unsure where to go from here. It doesn't seem appropriate to average the means and standard errors within each study. I'm also not incredibly familiar with the 'meta' package and can't find resources online detailing how to conduct a meta analysis with data like this. What should be my next step?


I'll try to address a few different components of your question

You seem to be under the impression that you want to meta-analyze means of some bird-related variable, but after reading...

I want to look at if, across birds, these response measures increase along with another variable that I have for each bird

it instead appears as though what you're really interested in meta-analyzing are the correlations between that bird-related variable, and some other variable that you have identified (potentially looking at differences across birds). Meta-analyzing correlations is certainly more typical than meta-analyzing means, though the latter is certainly possible--you can meta-analyze virtually any statistic with a corresponding variance or standard error. If it is, in fact, the case that you want to meta-analyze correlations between variables, then recording the means and standard errors of the one variable within each study won't help much. Instead, you will want to collect the correlations between the variables you are interested in, as well as the size of the sample for each correlation, as you'll use sample size to calculate the standard error and/or variance of each correlation.

Once you have this information, it appears as though you'll run into another issue: dependency of effect sizes. Meta-analysis, in many ways, is just a fancy weighted regression, and so many of the same assumptions apply--including assuming that all observations are independent. As you have indicated that...

Some studies use 10 response measures while others use one

..it seems likely that you will have some studies contributing multiple correlations, and therefore will be violating this assumption. Though it sounds scary (especially if you haven't don't meta-analysis before), you can use a method of 3-level meta-analysis (using multilevel structural equation models) to account for this dependency; this is easily accomplished using Cheung's metaSEM package for R (it's syntax is very similar to metafor and other meta-analysis packages). In effect, metaSEM allows you to specify a clustering variable that corresponds to how effect sizes (your correlations) are nested, so you could assign each study an ID number, and just use that as your clustering variable.

Bringing it all together, based on your description, it seems as though you would be interested in at least two models:

Model 1: An "intercept-only" model, whereby you estimate the meta-analytic average correlation between your two bird-related variables of interest.

Model 2: A model whereby you test if/how this correlation is moderated by some other variable(s) (e.g., type of bird, type of outcome measure used, etc.,).

The corresponding metaSEM code for each model is below. In the hypothetical example, you'd be using a data frame called "mydata", with columns called "ID", "corrs", "corrs_v", "bird_type", and "measure_type", corresponding to the ID # you assigned each study, the correlation(s) from the study, the variance of each correlation (metaSEM also allows you to specify standard errors instead), the type of bird, and the type of outcome measure respectively.

#Install and call metaSEM package

#Fit Model 1-Intercept-only
model.1=meta3(y = corrs, v = corrs_v, cluster = ID, data = mydata, model.name = "Intercept-only")

#Fit Model 2a-Moderation by Bird Type
model.2a=meta3(y = corrs, v = corrs_v, cluster = ID, data = mydata, x = cbind(bird_type), model.name = "Moderation by Bird Type")

#Fit Model 2b-Moderation by Outcome Measure Type
model.2a=meta3(y = corrs, v = corrs_v, cluster = ID, data = mydata, x = cbind(measure_type), model.name = "Moderation by Outcome Measure Type")

What's nice about the 3-level meta-analysis approach (and metaSEM) is that you will get the descriptive statistics of effect size variability that you normally would from a random-effects model (e.g., $\tau^2$, and $I^2$), except broken down for each level of clustering (i.e., one of each for within-study variability, and one for between study variability). Then, for your moderation models, you also get the benefit of having $R^2$ for each level of clustering, so you have an idea of how much variance in effect sizes your moderator(s) are explaining at each level.

So, to summarize, collect the effect sizes you are actually interested in meta-analyzing (it doesn't sound like you're interested in means, but rather, correlations of some sort) and their corresponding sample sizes (so you can calculate standard error or variance of the correlation; any introductory meta-analysis text will have these formulas). Then, once you have all this information entered, you can code an ID variable corresponding to which correlation(s) came from which study, and use metaSEM to estimate your meta-analytic model while appropriately accounting for the dependency among the effect sizes that come from the same study.

That should be plenty to get you started; if you have remaining questions, or if I misunderstood something, just comment this response and I can edit it as needed.

  • 1
    $\begingroup$ Thanks for this answer! It'll take me some time to thoroughly go over it but I wanted to make sure I understand one thing. What you're saying is that I'm really looking at how these bird variables correlate with my second variable? For example if my one variable I had for each bird was beak length, I'm looking at the correlations of whichever playback variables I have for each bird with beak length? Thanks! $\endgroup$ – user3403745 Feb 4 '16 at 14:12
  • 1
    $\begingroup$ I think so, but really, I can't/shouldn't tell you what effect you're interested in--that's something you need to decide for yourself. Meta-analysis is a technique for synthesizing studies on a particular effect, so one of the most foundational/important steps is deciding what effect you're interested in synthesizing in the first place. $\endgroup$ – jsakaluk Feb 4 '16 at 15:31
  • $\begingroup$ I've gone through your answer more thoroughly. What I am interested in is if the means from the playback studies increase with "beak length". The playback variables and beak length come from different studies so I'm not sure if I can calculate correlations for them. I guess what I'm trying to figure out is how to do the meta-analysis of either pooled means, or with means controlling for variable type (e.g. songs sung) and bird. I am interested in the correlations between beak length and response to playback but moreso in seeing if there is a correlation vs. using previous correlations. Thanks! $\endgroup$ – user3403745 Feb 4 '16 at 17:03
  • 1
    $\begingroup$ It sounds as though you are looking at a meta-regression. Just to clarify for each study you have one or more means and a beak length. If that is so then you could do meta-regression. I am not sure how to do that with metaSEM which I have never used so I will leave that for @jsakaluk. It is possible in rma.mv but since you have a very detailed example for metaSEM to build on it might be better to stick with that. No point changing horses in mid-stream. $\endgroup$ – mdewey Feb 4 '16 at 18:28

I think you need to use the metafor package and look at the function rma.mv and its documentation. It would also be worth looking at his website which has many examples. Perhaps you would like to try and then ask again? The author does post on Cross Validated but it may also be worth your while trying to post on the R-help mailing list. Sorry I cannot be too specific here but this is at the limits of my expertise.

There seems to have been a pause here so I will try to expand my answer with the benefit of the extra information which you have provided. I am going to suggest how to do this using metafor and the rma.mv function.

I assume you have all your variables in a data.frame and I have given them what I hope are obvious names below.

You start by specifying fit <- rma.mv(yi = outcome, V = se^2,

The you need to specify the random effects, which I think from your description will be random = ~ responsetype | study You need to make sure that responsetype is a factor or character variable.

You now need to specify your moderator variables with mods = ~ beaklength + responsetype This suppose that there is enough overlap between studies in the response types other wise you can only use beaklength here.

I would recommend setting slab = paste(study, responsetype) to get good labels in your forest plot. I would also strongly recommend using the profile function on the object to check the fit. With this sort of model it is sometimes the case that you do not have enough information in your data-set to identify the parameters. You may, as a biologist, be interested in this article by Nakagaw and Santos entitled "Methodological issues and advances in biological meta--analysis" which gives some of the theory. I think I downloaded my copy from the site of one of the authors in case you do not have access to that journal.

  • $\begingroup$ I don't see the need for a downvote here, but this should probably be a comment. $\endgroup$ – gung Feb 4 '16 at 0:55
  • $\begingroup$ Since it seems unlikely this helpful comment would expand into full answer I am inclined to move it to a comment. If you really think it does qualify as an answer or you edit it to more fully respond to the issues raised please flag this and ask for it to be restored as an answer. $\endgroup$ – Glen_b Feb 4 '16 at 3:18

You can use the standard mean difference for pooling different scales. for further reading check : 'metaanalysis with r' a book writen by the author of the package 'meta' pg 25

However, in many settings different studies use different outcome scales, e.g. different depression scales or quality of life scales. In such cases we cannot pool the effect estimates (mean differences) directly. Instead, we calculate a dimensionless effect measure from every study and use this for pooling. A very popular dimensionless effect measure is the standardised mean difference which is the study’s mean difference divided by a standard deviation based either on a single treatment group or both treatment groups.

He then used this command

#Ne,Nc sample size for both intervention,control,Me,Se for mean and standard deviation for each group, SMD for standardized mean difference
mc2 <- metacont(Ne, Me, Se, Nc, Mc, Sc, sm="SMD",
+ data=data2)

To get the result use summary(mc2), forest(mc2)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.