Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm doing some exploratory modelling on a data set with 29 covariates and an additional 11 variables that are of interest to my research question.

My strategy is to develop a model with a subset of the vector of covariates, and then evaluate model significance with the additional 11 research variables.

Irrespective of the specific modelling strategy, I'm afraid that I'm not sure how to deal with model uncertainty stemming from interpreting the AIC and BIC values I'm seeing.

In other words, if I calculate BIC for 5000 models, and find that the BIC is minimized (let's say = -1711) in a linear model with the variables A, B, C, and that the model with second-lowest BIC (let's say, = -1709) has the variables, A, B, C, and D -- how do I know that the difference between these two models is sufficient to justify excluding variable D from consideration? And how do I extend that examination of differences to consider the remaining 4,998 models?

Thanks in advance for any help you may be able to provide.

share|improve this question
As a rule of thumb, models with AIC differences $\leq 2$ are indistinguishable. See the wikipedia article on the AIC here: – Néstor Mar 6 '13 at 22:32
Thanks @Néstor for the helpful link. So I can use exp((AICmin−AICi)/2) as weights to calculate a multimodel? What are the implications of doing this with ~all~ available candidate models vs. only those where exp((AICmin−AICi)/2) > alpha? – dubhousing Mar 7 '13 at 18:36

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.