2
$\begingroup$

I am conducting a meta-analysis for proportions using the metafor-package in R and I have some trouble with the moderator analysis.

Basically, there are three aspects I'm not sure about:

  1. As the overall effect is calculated using transformations, and require a command to transform the effect size back to a proportion, should I do this as well for the moderators? (And if so, how?)
  2. I have included both categorical/dichotomous moderators as well as continuous moderators, do I need to specify this somewhere?
  3. I have added all the moderators into 1 model. Is this a good way of testing the moderators or should I have tested each moderator individually?

Code for overall effect:

Data <- escalc(measure="PFT", xi=Right, ni=N, data=Data)

res <- rma(yi, vi, method = "REML", data=Data) 

pred <- predict(res, transf=transf.ipft.hm, targs=list(ni=Data$N))

dat.back <- summary(Data, transf=transf.ipft, ni=N)

Output:

Random-Effects Model (k = 41; tau^2 estimator: REML)

tau^2 (estimated amount of total heterogeneity): 0.0122 (SE = 0.0028)
tau (square root of estimated tau^2 value):      0.1105
I^2 (total heterogeneity / total variability):   99.96%
H^2 (total variability / sampling variability):  2455.16

Test for Heterogeneity: 
Q(df = 40) = 113735.3842, p-val < .0001

Model Results:

estimate       se     zval     pval    ci.lb    ci.ub          
  1.0650   0.0176  60.6505   <.0001   1.0306   1.0995      *** 

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

   pred  ci.lb  ci.ub  cr.lb  cr.ub
 0.7659 0.7360 0.7945 0.5603 0.9211

Code for moderator analysis (Where cat = categorical moderator, dic = dichotomous moderator and con = continuous moderator):

res <- rma(yi, vi, mods = ~ Con1 + Con2 + Cat1 + Dic1 + Dic2 + Dic3 +
Dic4 + Dic5 + Dic6, method="REML", data=Data)

Output:

Mixed-Effects Model (k = 41; tau^2 estimator: REML)

tau^2 (estimated amount of residual heterogeneity):     0.0118 (SE = 0.0033)
tau (square root of estimated tau^2 value):             0.1088
I^2 (residual heterogeneity / unaccounted variability): 99.89%
H^2 (unaccounted variability / sampling variability):   911.72
R^2 (amount of heterogeneity accounted for):            3.07%

Test for Residual Heterogeneity: 
QE(df = 27) = 22291.5447, p-val < .0001

Test of Moderators (coefficient(s) 2,3,4,5,6,7,8,9,10,11,12,13,14): 
QM(df = 13) = 14.2799, p-val = 0.3544

Model Results:

                                estimate       se     zval    pval     ci.lb    ci.ub
intrcpt                           2.8676  12.3553   0.2321  0.8165  -21.3482  27.0835
Con1                             -0.0010   0.0062  -0.1579  0.8746   -0.0130   0.0111
Con2                              0.0046   0.0040   1.1509  0.2498   -0.0032   0.0123
Cat1A                            -0.0792   0.1664  -0.4759  0.6342   -0.4053   0.2470
Cat1B                             0.0508   0.2111   0.2404  0.8100   -0.3631   0.4646
Cat1C                            -0.1115   0.1837  -0.6067  0.5440   -0.4716   0.2486
Cat1D                             0.0517   0.1509   0.3428  0.7318   -0.2440   0.3474
Cat1E                            -0.0305   0.1606  -0.1897  0.8495   -0.3452   0.2843
Dic1                             -0.0145   0.0731  -0.1982  0.8429   -0.1578   0.1288    
Dic2                             -0.1013   0.1377  -0.7353  0.4622   -0.3712   0.1687
Dic3                              0.0332   0.0950   0.3498  0.7265   -0.1530   0.2194
Dic4                             -0.1281   0.1286  -0.9957  0.3194   -0.3802   0.1240
Dic5                              0.0915   0.0757   1.2091  0.2266   -0.0568   0.2398
Dic6                              0.0597   0.0572   1.0446  0.2962   -0.0524   0.1719                    

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 
$\endgroup$

1 Answer 1

2
$\begingroup$

Q1 if there is an easy way of transforming back onto the original scale then you can back transform the moderator coefficients but as far as I know this is not possible for the Freeman-Tukey one which you have used.

Q2 if you have your categorical moderator as a factor or character variable then R takes care of the issue for you. Note that you have more than one estimate for Cat.

Q3 testing the moderators one by one may tempt you into trying to do variable subset selection which generally disturbs the inference process in way s which are hard to predict so an overall model is usually preferred. Opinions do differ here and it does depend on your scientific question. There is an example in Wolfgang Viechtbauer's pages using the glmulti package http://www.metafor-project.org/doku.php/tips which may be helpful.

If this is your real data-set rather than a simulation then I would be concerned about (a) using so many moderators relative to the number of observation (b) the very high degree of heterogeneity even after fitting the moderators.

$\endgroup$
2
  • $\begingroup$ Thank you very much for taking the time to help me. Q1 I understand, so I would have to use the estimates that I reported in the output, but cannot say that Con1 reduces the effect size by .001? Q2 Thank you for the clarification. Regarding the moderators: I noticed the very high degree of heterogeneity as well. As the dataset involves studies that in most cases exceed N=10,000 or even N=100,000, could this explain the high heterogeneity or would it be related to the moderators I have coded for? (As the 95%CIs are incredibly small.) $\endgroup$
    – Tommy
    Commented Feb 13, 2017 at 20:09
  • $\begingroup$ Your suspicion is correct. If you have very precise primary studies you will ten to have very high heterogeneity. $\endgroup$
    – mdewey
    Commented Feb 13, 2017 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.