0
$\begingroup$

I want to perform a meta-analysis, for which I have data from 30 different experiments, each of them with two different treatments.

I have three different types experiments based on how the data are presented:

  • Experiments that have several years of data, and for each year there are 3 replicates per treatment. I guess in these cases I should calculate the mean value per replicate, and the overall mean per treatment would be the mean of the three replicates, and the SE will be based on the differences among replicates, hence n=3, regardless of the number of years?

    Experiment block year ambient elevated
    A 1 1998 234 397
    A 2 1998 209 395
    A 3 1998 254 347

    A 1 1999 234 317
    A 2 1999 249 390
    A 3 1999 204 348

  • Experiments with several years of data, containing mean and SE per year and treatment, but I don’t know how many replicates per treatment. How can I calculate the overall mean and SE per treatment at these experiments?

    Experiment year ambient ambient.SE elevated elevated.SE
    B 1998 234 12 397 32
    B 1999 209 14 395 23
    B 2000 254 20 347 18

  • Experiments that already have mean and SE per treatment calculated by others. All done.

    Experiment ambient ambient.SE elevated elevated.SE
    C 234 12 397 32

Based on this procedure, the weights for the meta-analysis (based on SE), basically give more importance to those experiments that have a response effect more similar among replicates, and the sample size. However, there is no consideration at all of the number of years that the experiments were carried out. My question is: can I calculate the mean for each experiment as the mean across years, so that the sample size is the number of years? Is this “legal” for a publication? Example: This way an experiment with data for 10 years would have more weight that an experiment which mean is based on 6 replicate measurements but only taken once after a few months of treatment. For my analysis makes sense that longer experiments are more “important” and hence have lower variance.

Thanks for your patience.

$\endgroup$
  • $\begingroup$ Are you looking at the difference between elevated and ambient (using Hedge's d or an equivalent)? I think the answer to your question is yes, but you could use Borenstein et al equations to combine dependent and independent effect sizes onlinelibrary.wiley.com/book/10.1002/9780470743386. $\endgroup$ – Emilie Jul 6 '15 at 12:42
  • $\begingroup$ Yes, the effect ratio is elev/amb $\endgroup$ – fede_luppi Jul 6 '15 at 13:29
1
$\begingroup$

My question is: can I calculate the mean for each experiment as the mean across years, so that the sample size is the number of years?

I would say yes, but you need to use the proper equations that will adjust the variance and thus the weight of each effect size.

The following answer is bases on Borenstein, M., L. V. Hedges, J. P. T. Higgins et H. R. Rothstein (2009). Introduction to meta-analysis. Chichester, UK, John Wiley & Sons, Ltd.

Some experiment report multiple outcomes. Those outcomes can be independent, like the blocks in your first example. Other outcomes are dependent, like multiple measurements taken years after years in a single plot.

  • Independent outcomes: you can treat each subgroup as a separate study (each block will have an effect size in the analysis) or you can compute a composite effect size. You compute that summary effect size by doing a fixed-effect meta-analysis on the blocks of that study. You will use the resulting mean outcome and variance for your multi-study meta-analysis. Another option is to recreate the summary data for the full study (but I cannot developp more this option without more details on your data).
  • Dependent outcomes: A little trickier. Here's the equation you should use to combine the outcomes of multiple studies (sorry I'm not sure I add the equation properly):

enter image description here

where Y is an effect size and m the number of outcomes in the study.

The variance is calculated as

enter image description here

were r is the correlation between the outcomes. The thing is...we don't know the correlation in most case ! You could be super conservative and say the correlation is perfect (r = 1) or be super liberal and say there is no correlation between years (r = 0). It all depends on your study system and the existence of known correlation.

Here's what I did in a similar situation: My data were biological data, so the correlation is probably not 0. I thus used r = 1 and did my meta-analysis. I recombined all effects sizes with r = 0 and redid the meta-analysis to see the impact of r's choice on my results. I'm lucky, the arbitrary choice of r value did not changed my results, so I can say my results are robust to the assumption of perfect correlation between outcomes.

I hope this will help you.

Edit: I forgot some explanation in the variance equation.

$\endgroup$
  • $\begingroup$ Dear Emilie, thanks for your answer. I use the R package {metafor} for meta-analysis, and I was hoping I could transform the different types of outcomes so all can be presented in the same format and then run metafor to calculate effect sizes and variance. For the independent outcomes calculating mean and SE is straight-forward (average the effect among blocks for each experiment), but not for the dependent outcomes. In the second example of outcome, I can average the means, but can I simply average SE values as well for each experiment? $\endgroup$ – fede_luppi Jul 6 '15 at 14:25
  • $\begingroup$ I used 'metafor' too but as it was my first meta-analysis and I had a lot of outcomes to combine, I did it by hand in Excel ! I think a simple average of SE cannot be done, because the formula need the correlation between the outcomes (I will clarify my answer). $\endgroup$ – Emilie Jul 6 '15 at 15:06
  • $\begingroup$ Emilie, for each study and year I have mean and SE. In order to calculate the varience from SE values to use the equation above I guess I need sample sizes? The thing is that I don’t always know sample sizes for some studies... $\endgroup$ – fede_luppi Jul 7 '15 at 9:30
  • $\begingroup$ This goes beyond a simple comment. I suggest you take the time to read the corresponding chapter of Borenstein for more details. Also, if the answer is correct to you, don't forget to accept it ! $\endgroup$ – Emilie Jul 7 '15 at 13:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.