6
$\begingroup$

I've got a set of multivariate regression models, with weights, that I'm trying to compare in R. Looks like:

f0 <- lm(cbind(Y1,Y2,Y3,Y4) ~ co1 + co2 + co3 + co4 + x1, dat, weight=wt)
f1 <- lm(cbind(Y1,Y2,Y3,Y4) ~ co1 + co2 + co3 + co4 + x1 + x2, dat, weight=wt)
f2 <- lm(cbind(Y1,Y2,Y3,Y4) ~ co1 + co2 + co3 + co4 + x1 + poly(x2,2), dat, weight=wt)

I get perfectly reasonable and interpretable fits, but I'd like to be able to answer a question like, "overall, is there an effect of x2 on the dependent measures?" In other contexts, with nested models, I've done something like:

anova(f0, f1, f2)

and used a Chi-square test. But in this case, I get this error:

Error in SSD.mlm(object) : 'mlm' objects with weights are not supported

So, what alternatives do I have to compare these models? Thanks!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

I recommend calculating AIC model weights for each of these. This gives you the flexibility of being able to choose the AIC-best model for inference, or if there is considerable structural uncertainty, to obtain a model-weighted estimate of the effect of x2.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks -- to clarify, you're recommending I pick the model with the best AIC, then report the fitted coefficients of that model? In this case I get AIC-deltas of 5, 0, and 1.5, respectively. Is there a test that says that f1 is better than f0, through? That's what I would have gotten if anova() had worked... $\endgroup$
    – Harlan
    Commented Feb 27, 2012 at 16:37
  • $\begingroup$ You can convert the delta AIC values to model weights (probabilities) via: $$ p_i = \frac{\exp(-0.5 \Delta AIC_i)}{\sum_j \exp(-0.5 \Delta AIC_j)} $$ So, you can use those probabilities to more easily decide which models are relevant. Based on those values, it may be worth estimating a model-averaged coefficient, since your second and third models appear to be close in AIC distance. That way you do not have to worry about model selection uncertainty (i.e. throwing out the wrong model). I recommend Burnham and Anderson 2002 as a good reference. $\endgroup$
    – fonnesbeck
    Commented Feb 27, 2012 at 19:02
  • $\begingroup$ Yes, I have Burnham and Anderson, and after you posted your answer, I flipped through it and found exactly the formula above! Thanks, this looks like a pretty good option! $\endgroup$
    – Harlan
    Commented Feb 27, 2012 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.