I am confused and just need some confirmation about calculating the relative variable importance value for the co-variates I used in AIC model selection procedures. I know that there is this one discussion but it doesn't quite confirm explicitly enough what I should do.
Burnham and Anderson (2002) describe a simple way to quantify variable importance.
Page 168: Estimates of the relative importance of predictor variables xj can best be made by summing the AIC weights across all the models in the set where variable j occurs.
However, to use this method, one must have an equal number of models for each variable; otherwise, some variables will be over represented or under represented resulting in biased relative importance values.
Page 169: When assessing the relative importance of variables using sums of the AIC weights, it is important to achieve a balance in the number of models that contain each variable j.
Does this mean if I have a set of models with their model weights from AIC procedure (these are not ranked by weight, just the order I created them):
1 INTERCEPT
2 REPRO TIME
3 REPRO TIME R*T
4 REPRO TIME WR
5 REPRO TIME WR WR*R
6 REPRO TIME WR WR*T
7 WR
To calculate the relative variable weight I would sum up the weight for each incident that TIME was in the models and I would do so for each of the other variables. However, this is not completely correct right? Because there is not a balance in the number of models that contain each variable right? So, to correct for this I would then divide the sum of these weights by the number of models that had that variable. (Kittle et al 2008 "the scale-dependent impact of wolf predation risk...." does this). So, for instance if the sum of the weights for time was .75 I would divide it by 5 because it was in 5 of the models, likewise, WR would be divided by 4.
It seems like a silly question but it really changes the results and interpretation from my analysis. Because for instance, WRT is only in 1 model and it comes out to be in one of the top models so it has a high model weight, but Time and Repro are also in this top model but also in 4 other candidate models. So dividing the weight of T & R by 5 reduces the importance of T or R from (0.999), giving them a RVI of 0.2 and the RVI to WRT value of 0.7. Is that right?
In addition to this, my next question would be - do you do this over JUST the "BEST" (within 2AIC or what ever criteria) models or over all 7 regardless of what surfaced to the top? I used MuMIn package and use the importance command, but then when you use the get best models, it asks if you want to recalculate the importance which then it recalculates for just the top models. Which is more appropriate to use? This doesn't make sense when only 1 model is the best. I would then assume it should be calculated over all models.