Difference in AIC as a measure of relative importance of variables

Question

I wonder whether it is sensible to use the following method of assessing relative importance of variables in a model.

1. Compute a model and its $AIC$. Let's say it has $k$ predictors.
2. Compute $k$ submodels, each of which has one predictor removed. Compute $AIC$ for all the submodels.
3. Compute $k$ times $\Delta AIC$, which is defined as $AIC_{submodel} - AIC_{full~model}$.
4. And then the relative importance of each variable is being quantified by the magnitude of its corresponding $\Delta AIC$.

EDIT. Just to clarify a few things: this approach is meant to be used in a specific situation, when one has a bunch of otherwise equally important predictors (that is, none of them is considered more important than others a priori) and want to figure out which of the predictors has the strongest relationship with the outcome. The idea is that the difference in AIC quantifies relative likelihood that one model minimizes information loss and not the other, so having $\Delta AIC$ defined for all predictors one can judge which of them minimize information loss the most.

So the aim here is to establish a relative hierarchy of predictors within a single model. Does this strategy make any sense?

Answer 1

Burnham and Anderson have offered some guidelines on this. The best idea would probably be to look at their text book about information-based statistical inference:

Burnham, K. P.; Anderson, D. R. (2002), Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach (2nd ed.), Springer-Verlag, ISBN 0-387-95364-7.

But I believe they talk about guidelines of using gaps of 2 as providing some evidence, and 4 providing stronger evidence, to prefer the larger model. There are more levels of evidence beyond that. They talk about this in various places, e.g. this paper:

http://www.ericlwalters.org/Burnham_etal_2011.pdf

Answer 2

In addition to the great points @justanotherbrain mentioned, I'd like to add that "importance" of a variable is quite subjective and really depends on the application of the model. If it's an explanatory model where you are trying to make an inference on the association between an explanatory variable and the outcome, it's quite likely that some variables included in the model may not contribute very much in dropping the AIC. For example, if you want to test the association between a drug and a health outcome, but the drug is ineffective, the "drug" variable remaining in the model is unlikely to affect the AIC by much, but it is it the most important explanatory variable in the model.

In the case of predictive models, again, AIC may not be the appropriate metric to use, as each model that you build should be optimized specifically to the application of the model as well as the type of model being built. A model of continuous outcomes will have very different model fit statistics than a model of discrete outcomes.

There are actually existing well-established methods which may accomplish what you want. The LASSO developed by Rob Tibshirani is one where you could establish some sort of "variable importance hierarchy", and a more flexible approach then the one you describe, since each variable's parameter is regularized, rather than the binary status of being "in" or "out" of the model.