I am performing a meta-analysis in R with metafor
package. Primary studies report the reduction of plant species to certain amount of nitrogen fertilization. Effect size is calculated as the ratio of number of species in the treatment group divided the control for each observation.
Following Borenstein (2009), there is no reason to believe a fixed effect model should be used since these experiments are all undertaken in different ecosystems, vegetation types, soil type et cetera.
However, heterogeneity of the pooled effect sizes obtained using random models of rma()
is extremely low.
Total heterogeneity is very low (I^2 = 0.00%) for regressions fitted with "DL" method:
> res_means <- rma(S_ratio, Variance, data = dat_S_ratio, method="DL")
> res_means
Random-Effects Model (k = 236; tau^2 estimator: DL)
tau^2 (estimated amount of total heterogeneity): 0 (SE = 0.0090)
tau (square root of estimated tau^2 value): 0
I^2 (total heterogeneity / total variability): 0.00%
H^2 (total variability / sampling variability): 1.00
Test for Heterogeneity:
Q(df = 235) = 170.8544, p-val = 0.9994
Model Results:
estimate se zval pval ci.lb ci.ub
0.7403 0.0203 36.5345 <.0001 0.7005 0.7800 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Even using REML (I read somewhere that is actually more accurate), heterogeneity is still very low (I^2 = 10.99%):
> res_means <- rma(S_ratio, Variance, data = dat_S_ratio, random=~1|PrimaryStudy, control=list(stepadj=.5))
> res_means
Random-Effects Model (k = 236; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0.0120 (SE = 0.0090)
tau (square root of estimated tau^2 value): 0.1096
I^2 (total heterogeneity / total variability): 10.99%
H^2 (total variability / sampling variability): 1.12
Test for Heterogeneity:
Q(df = 235) = 170.8544, p-val = 0.9994
Model Results:
estimate se zval pval ci.lb ci.ub
0.7628 0.0221 34.4510 <.0001 0.7194 0.8062 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Subgroup analysis provide also very low values of heterogeneity when effect size is calculated for each subgroup.
Is it possible to have such unexpected value of heterogeneity? I suspect this is due to the way I calculated the variance but I am not sure about it. Indeed variance was not extracted from primary studies but back-calculated with the following for each observation:
Variance calculation obtained with the Delta method, assuming data are following a Poisson distribution. Sst is the richness of species in the treatment group, Ssc is the richness of species at the control group
Looking at the forest plot, confidence intervals looks similar to each other in terms of range (ok, there are differences but not THAT big), although for each observation the effect size is different. Is this an explanation why heterogeneity is so low?
Does it make sense to still perform a meta-regression on observation with such low heterogeneity?
S_ratio
) is a count divided by another count? Then I would suggest to log transform those values. Let's say the ratios in two studies are 20/10 and 10/20. Then the average would be 1.25, which makes little sense. Ratios of counts are not symmetric around 1. Log transforming the ratios solves this issue. Also, log transforming should make the sampling distribution more normal (an assumption underlying the model you are using). The variance is then $1/S_{st} + 1/S_{sc}$. $\endgroup$