# How to compare weighted multivariate linear models?

I've got a set of multivariate regression models, with weights, that I'm trying to compare in R. Looks like:

f0 <- lm(cbind(Y1,Y2,Y3,Y4) ~ co1 + co2 + co3 + co4 + x1, dat, weight=wt)
f1 <- lm(cbind(Y1,Y2,Y3,Y4) ~ co1 + co2 + co3 + co4 + x1 + x2, dat, weight=wt)
f2 <- lm(cbind(Y1,Y2,Y3,Y4) ~ co1 + co2 + co3 + co4 + x1 + poly(x2,2), dat, weight=wt)


I get perfectly reasonable and interpretable fits, but I'd like to be able to answer a question like, "overall, is there an effect of x2 on the dependent measures?" In other contexts, with nested models, I've done something like:

anova(f0, f1, f2)


and used a Chi-square test. But in this case, I get this error:

Error in SSD.mlm(object) : 'mlm' objects with weights are not supported


So, what alternatives do I have to compare these models? Thanks!

• Thanks -- to clarify, you're recommending I pick the model with the best AIC, then report the fitted coefficients of that model? In this case I get AIC-deltas of 5, 0, and 1.5, respectively. Is there a test that says that f1 is better than f0, through? That's what I would have gotten if anova() had worked... – Harlan Feb 27 '12 at 16:37
• You can convert the delta AIC values to model weights (probabilities) via: $$p_i = \frac{\exp(-0.5 \Delta AIC_i)}{\sum_j \exp(-0.5 \Delta AIC_j)}$$ So, you can use those probabilities to more easily decide which models are relevant. Based on those values, it may be worth estimating a model-averaged coefficient, since your second and third models appear to be close in AIC distance. That way you do not have to worry about model selection uncertainty (i.e. throwing out the wrong model). I recommend Burnham and Anderson 2002 as a good reference. – fonnesbeck Feb 27 '12 at 19:02