1
$\begingroup$

I am a beginner in statistics and currently have to conduct a meta-analysis in my lab. I am facing some difficulties to understand the meta-regression.

I read this webpage and tried to follow along: http://www.metafor-project.org/doku.php/tips:testing_factors_lincoms

This is the code that I tried:

library(metafor)
dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)

res <- rma(yi, vi, mods = ~ factor(alloc), data=dat)
res

res <- rma(yi, vi, mods = ~ factor(alloc) - 1, data=dat)
res

Results:

Mixed-Effects Model (k = 13; tau^2 estimator: REML)

tau^2 (estimated amount of residual heterogeneity):     0.3615 (SE = 0.2111)
tau (square root of estimated tau^2 value):             0.6013
I^2 (residual heterogeneity / unaccounted variability): 88.77%
H^2 (unaccounted variability / sampling variability):   8.91
R^2 (amount of heterogeneity accounted for):            0.00%

Test for Residual Heterogeneity: 
QE(df = 10) = 132.3676, p-val < .0001

Test of Moderators (coefficient(s) 2:3): 
QM(df = 2) = 1.7675, p-val = 0.4132

Model Results:

                         estimate      se     zval    pval    ci.lb   ci.ub   
intrcpt                   -0.5180  0.4412  -1.1740  0.2404  -1.3827  0.3468   
factor(alloc)random       -0.4478  0.5158  -0.8682  0.3853  -1.4588  0.5632   
factor(alloc)systematic    0.0890  0.5600   0.1590  0.8737  -1.0086  1.1867   

--- 

Mixed-Effects Model (k = 13; tau^2 estimator: REML)

tau^2 (estimated amount of residual heterogeneity):     0.3615 (SE = 0.2111)
tau (square root of estimated tau^2 value):             0.6013
I^2 (residual heterogeneity / unaccounted variability): 88.77%
H^2 (unaccounted variability / sampling variability):   8.91

Test for Residual Heterogeneity: 
QE(df = 10) = 132.3676, p-val < .0001

Test of Moderators (coefficient(s) 1:3): 
QM(df = 3) = 15.9842, p-val = 0.0011

Model Results:

                         estimate      se     zval    pval    ci.lb    ci.ub     
factor(alloc)alternate    -0.5180  0.4412  -1.1740  0.2404  -1.3827   0.3468     
factor(alloc)random       -0.9658  0.2672  -3.6138  0.0003  -1.4896  -0.4420  ***
factor(alloc)systematic   -0.4289  0.3449  -1.2434  0.2137  -1.1050   0.2472 

Questions:

  1. In my understanding, the first model has an intercept (the reference level) while the second model leaves out the intercept. But why would this yield different results in the omnibus test? The first one showed insignificance and the second one showed significance. So, is this moderator significant at all?

  2. In the first model, the "estimate" column in the "Model Results" is how much the estimated effect size of random allocation and systematic allocation is higher/lower, compared to the alternate allocation (reference level). Then what does the p-value mean? Is it whether the systematic allocation is significantly higher/lower compared to the reference level? But how about when the intercept has a significant p-value? What does that mean?

  3. In the second model, the "estimate" column in the "Model Results" is how much the estimated effect size of alternate allocation, random allocation and systematic allocation respectively. Then what does the p-value mean here? Is it whether the estimate is significantly different from 0?

  4. Can we change the reference level from alternate allocation to systematic allocation or random allocation?

$\endgroup$
  • $\begingroup$ Hi Beginner, welcome to CV! Note that 1) there's no need to write thanks at the end of the post, and more importantly 2) at the moment, your question is extra broad. The best practice here is to ask a single question per thread and narrow it down as possible. $\endgroup$ – Jan Kukacka Jun 14 '18 at 9:55
  • $\begingroup$ Hi Jan, do you think I should separate the questions to different threads? I think the questions I ask is related to each other that they are all about how I should interpret the "Model Result" given by the metafor function. $\endgroup$ – Stat Beginner Jun 14 '18 at 10:07
  • $\begingroup$ Yes, I would suggest asking rather several focused questions that have potential also to help other people in the future. At the present, you are basically asking others to do the data analysis for you, which is not the main purpose of this site. $\endgroup$ – Jan Kukacka Jun 14 '18 at 10:13
  • $\begingroup$ Hm.. I think what I was asking is more about the conceptual issue rather than trying to ask others to help me do the analysis (as my one is much more complicated than the example here, which is a multilevel meta-analysis). The questions that I asked here is something that really confused me and I couldn't find the answer by searching through the net... as it seems that all statisticians took this conceptual questions for granted, thinking that everyone would know and just jump through it real quick $\endgroup$ – Stat Beginner Jun 14 '18 at 10:19
  • $\begingroup$ @JanKukacka Also, in my post, I have mentioned in details what my thoughts are. My question is basically whether my guess is a correct one. (e.g. does the p-value really means that allocation is sigificantly higher/lower than the reference level??). It's a really crass remark to say that I am asking others to do the analysis for me. Can you point out specifically which part of my post is asking for that? $\endgroup$ – Stat Beginner Jun 14 '18 at 10:21
1
$\begingroup$

It might have been better to ask separately but here goes an attempt

Q1 you are testing completely different hypotheses. The first is that the two relevant coefficients are both zero, the second model that all three coefficients are zero.

Q2 yes it does mean they are different from the reference category which may or may not itself be different from zero

Q3 yes

Q4 yes, but how is really an R programming question is it not?

$\endgroup$
  • $\begingroup$ Thanks a lots!! The Q1 and Q2 is the one that I specifically want to confirm. $\endgroup$ – Stat Beginner Jun 14 '18 at 10:47

Not the answer you're looking for? Browse other questions tagged or ask your own question.