Testing the effect of moderators in metafor package

Question

I am conducting multilevel meta-analyses in R using the metafor package. I have 177 treatment vs. control comparisons from the data from 30 studies (multiple rows per study due to multi-year studies). Thus I am using random effects model rma.mv by keeping study id (reference or citation name) and year (the year in which observation was taken) as random effects (random=~1|id/year). I also want to see fixed effects of two moderators (crop and soil). Moderator “crop” has 7 levels and moderator “soil” has 3 levels. I ran first model (Model1) using rma.mv function, to see the overall effect of treatment. Then I ran three other models to test the effect of two moderators (please see syntax). The QM test is significant only for Model2 and Model4. Then I compared all four models using AIC criteria, that showed Model1 (and Model2?) is the best among all, however it does not include any moderator. AIC for Model1, 2, 3 and 4 are -77.07, -75.82, -56.95 and -58.28, respectively.

My questions are:

1. If I choose Model1, according to AIC, then how could I test and discuss the effects of crop and soil types on response variable?
2. When I use rma function, the results differ compared to that when I use rma.mv, in terms of significance. Which function should I choose?

Thank you in advance.

Model1<-rma.mv(yi, vi, random=~1|id/year, data=b)

Model2<-rma.mv(yi, vi, mods = ~ factor(soil)- 1, random=~1|id/year, data = a)
Model3<-rma.mv(yi, vi, mods = ~ factor(crop) - 1, random=~1|id/year, data = a)

Model4<-rma.mv(yi, vi, mods = ~ factor(soil)+factor(crop) - 1, random=~1|id/year, data = a)

Answer 1

Comparisons of information criteria only make sense under maximum likelihood estimation. [1] So if you are going to compare AIC values, fit the models with method="ML".

Also, you remove the intercept term in models 2, 3, and 4. This does not tell you if the factor is statistically significant (see: https://www.metafor-project.org/doku.php/tips:models_with_or_without_intercept). If you want to test if there are differences between the levels of a factor, leave the intercept in the model.

As for your second question: rma.mv() since your data has a multilevel structure, which rma() would not account for.

[1] There is actually some support for possibly comparing information criteria under REML estimation (https://doi.org/10.1198/000313006X90396). However, this is a contentious issue and to what extent the results from Gurka (2006) transfer to other types of models remains to be examined.