Skip to main content
Removed random-effects tag
Link
mdewey
  • 18.4k
  • 23
  • 35
  • 61
added the anova code
Source Link
Johanna
  • 491
  • 3
  • 13

I am running a simple random-effects meta-analysis in R using the metafor package, with random intercepts at the study level:

mod1 <- rma.mv(Hedges_g, cov, random = ~ 1 | study, data = rev)

This is the model output:

Multivariate Meta-Analysis Model (k = 90; method: REML)

   logLik   Deviance        AIC        BIC       AICc  
-170.3401   340.6802   344.6802   349.6575   344.8197  

Variance Components: 

            estim    sqrt  nlvls  fixed  factor
sigma^2    0.5512  0.7424     24     no   study

Test for Heterogeneity: 
Q(df = 89) = 1014.3323, p-val < .0001

Model Results:

estimate       se     zval     pval    ci.lb    ci.ub          
  0.9749   0.1572   6.2018   <.0001   0.6668   1.2830

As I understand, the observation of significant heterogeneity means that the estimate of g = 0.97 cannot be regarded as an estimate of one true effect. Rather, the studies in this data set seem to be estimating different true effects.

Now, I'm comparing my model (mod1) to another model without random intercepts at the study level: mod0 <- rma.mv(Hedges_g, cov, data = rev, method = "ML") (I have set method = "ML" for mod1 too, to enable the comparison). This is the output for anova(mod0, mod1):

        df      AIC      BIC     AICc    logLik      LRT   pval        QE
Full     2 347.2963 352.2960 347.4343 -171.6482                 1014.3323
Reduced  1 916.1363 918.6361 916.1818 -457.0682 570.8400 <.0001 1014.3323

Thus, mod1 fits the data significantly better than mod0. This means that the estimated between-study variance of sigma^2 = 0.55, is significant. To me, this would also suggest that the studies are estimating significantly different true effects.

My question now is: what is the difference between the test for heterogeneity, and the model comparison? Do they both lead to the exact same conclusion ("There is heterogeneity among the true effects"), or is there more nuance to it?

I am running a simple random-effects meta-analysis in R using the metafor package, with random intercepts at the study level:

mod1 <- rma.mv(Hedges_g, cov, random = ~ 1 | study, data = rev)

This is the model output:

Multivariate Meta-Analysis Model (k = 90; method: REML)

   logLik   Deviance        AIC        BIC       AICc  
-170.3401   340.6802   344.6802   349.6575   344.8197  

Variance Components: 

            estim    sqrt  nlvls  fixed  factor
sigma^2    0.5512  0.7424     24     no   study

Test for Heterogeneity: 
Q(df = 89) = 1014.3323, p-val < .0001

Model Results:

estimate       se     zval     pval    ci.lb    ci.ub          
  0.9749   0.1572   6.2018   <.0001   0.6668   1.2830

As I understand, the observation of significant heterogeneity means that the estimate of g = 0.97 cannot be regarded as an estimate of one true effect. Rather, the studies in this data set seem to be estimating different true effects.

Now, I'm comparing my model (mod1) to another model without random intercepts at the study level: mod0 <- rma.mv(Hedges_g, cov, data = rev, method = "ML") (I have set method = "ML" for mod1 too, to enable the comparison). This is the output:

        df      AIC      BIC     AICc    logLik      LRT   pval        QE
Full     2 347.2963 352.2960 347.4343 -171.6482                 1014.3323
Reduced  1 916.1363 918.6361 916.1818 -457.0682 570.8400 <.0001 1014.3323

Thus, mod1 fits the data significantly better than mod0. This means that the estimated between-study variance of sigma^2 = 0.55, is significant. To me, this would also suggest that the studies are estimating significantly different true effects.

My question now is: what is the difference between the test for heterogeneity, and the model comparison? Do they both lead to the exact same conclusion ("There is heterogeneity among the true effects"), or is there more nuance to it?

I am running a simple random-effects meta-analysis in R using the metafor package, with random intercepts at the study level:

mod1 <- rma.mv(Hedges_g, cov, random = ~ 1 | study, data = rev)

This is the model output:

Multivariate Meta-Analysis Model (k = 90; method: REML)

   logLik   Deviance        AIC        BIC       AICc  
-170.3401   340.6802   344.6802   349.6575   344.8197  

Variance Components: 

            estim    sqrt  nlvls  fixed  factor
sigma^2    0.5512  0.7424     24     no   study

Test for Heterogeneity: 
Q(df = 89) = 1014.3323, p-val < .0001

Model Results:

estimate       se     zval     pval    ci.lb    ci.ub          
  0.9749   0.1572   6.2018   <.0001   0.6668   1.2830

As I understand, the observation of significant heterogeneity means that the estimate of g = 0.97 cannot be regarded as an estimate of one true effect. Rather, the studies in this data set seem to be estimating different true effects.

Now, I'm comparing my model (mod1) to another model without random intercepts at the study level: mod0 <- rma.mv(Hedges_g, cov, data = rev, method = "ML") (I have set method = "ML" for mod1 too, to enable the comparison). This is the output for anova(mod0, mod1):

        df      AIC      BIC     AICc    logLik      LRT   pval        QE
Full     2 347.2963 352.2960 347.4343 -171.6482                 1014.3323
Reduced  1 916.1363 918.6361 916.1818 -457.0682 570.8400 <.0001 1014.3323

Thus, mod1 fits the data significantly better than mod0. This means that the estimated between-study variance of sigma^2 = 0.55, is significant. To me, this would also suggest that the studies are estimating significantly different true effects.

My question now is: what is the difference between the test for heterogeneity, and the model comparison? Do they both lead to the exact same conclusion ("There is heterogeneity among the true effects"), or is there more nuance to it?

Source Link
Johanna
  • 491
  • 3
  • 13

Meta-analysis: significant heterogeneity vs. significant between-study variance

I am running a simple random-effects meta-analysis in R using the metafor package, with random intercepts at the study level:

mod1 <- rma.mv(Hedges_g, cov, random = ~ 1 | study, data = rev)

This is the model output:

Multivariate Meta-Analysis Model (k = 90; method: REML)

   logLik   Deviance        AIC        BIC       AICc  
-170.3401   340.6802   344.6802   349.6575   344.8197  

Variance Components: 

            estim    sqrt  nlvls  fixed  factor
sigma^2    0.5512  0.7424     24     no   study

Test for Heterogeneity: 
Q(df = 89) = 1014.3323, p-val < .0001

Model Results:

estimate       se     zval     pval    ci.lb    ci.ub          
  0.9749   0.1572   6.2018   <.0001   0.6668   1.2830

As I understand, the observation of significant heterogeneity means that the estimate of g = 0.97 cannot be regarded as an estimate of one true effect. Rather, the studies in this data set seem to be estimating different true effects.

Now, I'm comparing my model (mod1) to another model without random intercepts at the study level: mod0 <- rma.mv(Hedges_g, cov, data = rev, method = "ML") (I have set method = "ML" for mod1 too, to enable the comparison). This is the output:

        df      AIC      BIC     AICc    logLik      LRT   pval        QE
Full     2 347.2963 352.2960 347.4343 -171.6482                 1014.3323
Reduced  1 916.1363 918.6361 916.1818 -457.0682 570.8400 <.0001 1014.3323

Thus, mod1 fits the data significantly better than mod0. This means that the estimated between-study variance of sigma^2 = 0.55, is significant. To me, this would also suggest that the studies are estimating significantly different true effects.

My question now is: what is the difference between the test for heterogeneity, and the model comparison? Do they both lead to the exact same conclusion ("There is heterogeneity among the true effects"), or is there more nuance to it?