1
$\begingroup$

This is in continuation of my previous post (Testing the effect of moderators in metafor package) Thanks@Wolfgang for the clarification. Now I used model: A<-rma.mv(yi, vi, mods = ~ factor(soil)+factor(crop)+ ppt, random=~1|id/studyyear, method = "REML", data = a) [I also added precipitation (ppt) now; as a continuous variable]

My questions are:

  1. How do we extract overall effect of treatment from this model ? (I extracted the overall effect by deleting all the moderators, but wondering if I could get overall effect from this model including moderators?)
  2. Also the intercept includes both 1 level of soil and 1 level of crop as a reference, I am confused that how do I compare other levels of soil to the intercept since intercept is also acting as reference for crop factor at same time. Is there any way to separate these two? When I delete crop from model, intercept value changes and becomes significant (in this case, does it mean that that 1 soil level is significantly different from zero?). I want to look avg values for each of factor levels and whether they significantly differ from zero. How could I do that?
  3. I have named individual study ids for each study in the dataset, but a couple of studies are actually conducted on the same field sites. So Should the "id" in my model be the study or the site (and thus do I have to rename ids in the dataset according to sites rather than according to studies)?
  1. The total number of studies in my dataset are 30, which seems like at the borderline. So if the effect of any factor level comes out as non-significant, does it mean that it is truly non-significant or it is due to the lack of enough data. In other words, How can we calculate the power in drawing conclusions? (Non-significance = underpowered vs. truly non-significant)? I am also testing another response variable for which the no. of studies are only 8. Same question goes for it or should I even conduct the analysis for that at all?

I would really appreciate if you could guide me on this. Thanks.

$\endgroup$
1
$\begingroup$

Q1 - if you show an effect of the moderator then there is no overall effect. It would be like estimating the average height of humans when you have shown that men are taller than women.

Q2 - the coefficient for crop applies irrespective of soil level and vice versa unless you fit an interaction.

Q3 - with the sort of numbers you quote I suspect if you try to fit random effects at the site level and at the study level one of them will be based on little data and be quite unstable so I would plump for a study effect.

Q4 - with 30 observations I would not want to use more than 2 degrees of freedom for my moderators or perhaps 3 if pushed.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.