Discrepancy between metafor and weighted lm() standard errors

Question

My friend and I are trying to understand why a weighted lm() v. a fixed-effect rma model from the metafor package are producing identical meta-analytic point estimates, but different standard errors (and therefore p values and 95% CIs) for those estimates.

For example:

#install and call metafor package
install.packages("metafor")
library(metafor)

#read in example data for standardized mean differences and standard errors
d<-c(0.38, 0.36, -0.35, 1.55, 0.26, 1.2, 0.38, 0.46, 0.27, 0.24, -0.07, -0.26, -0.31, 1.15, 0.23, 0.29, 0.38, 0.19, 0.4, 0.15, 0.2, 0.25, 0.34)
d.se<-c(1.8, 3.49, 1.53, 4.96, 2.08, 3.48, 0.07, 0.07, 0.09, 0.09, 0.01, 0.09, 6.64, 5.08, 7.44, 0.16, 0.18, 2.05, 0.17, 0.16, 0.17, 0.22, 0.09)
d.v<-d.se*d.se

#run fixed-effects intercept-only models with lm() and metafor
lm.intercept<-lm(d ~ 1, weights=I(1/d.v))
summary(lm.intercept)

metafor.intercept<-rma(yi=d, vi=d.v, method="FE")
summary(metafor.intercept)

Both approaches yield the appropriate estimate of -0.0361, but the lm() approach yields a standard error of 0.02585 (and therefore the estimate is not significant at the p >.05 level), whereas the metafor approach yields a standard error of 0.0095 (and therefore the estimate is significant at the p < .05 level). The same discrepancy also occurs for moderators that you add to the model (e.g., d~d.se).

I am somewhat confident that the lm() approach is mistakenly estimating the standard error somehow (I've worked through the calculations for this example by hand: google doc spreadsheet here), but my friend and I would like to better understand why/how this is occurring.

Does anyone have any idea of what's causing the discrepancy in standard errors between the two models?

Answer 1

A while ago, I wrote up an extensive comparison between the rma() function from the metafor package and the lm() and lme() functions (the latter from the nlme package) for fitting fixed- and random/mixed-effects models. You can find this on the metafor package website:

http://www.metafor-project.org/doku.php/tips:rma_vs_lm_and_lme

To briefly summarize: When you use the lm() and lme() functions with weights, then this fits models that assume that the weights (i.e., sampling variances) are known only up to a proportionality constant -- which is in fact the error variance that is estimated. Those are not standard meta-analytic models as commonly described in the literature.