AIC stands for the Akaike Information Criterion, which is one technique used to select the best model from a class of models using a penalized likelihood. A smaller AIC implies a better model.
66
votes
10answers
13k views
Is there any reason to prefer the AIC or BIC over the other?
The AIC and BIC are both methods of assessing model fit penalized for the number of estimated parameters. As I understand it, BIC penalizes models more for free parameters than does AIC. Beyond a ...
20
votes
8answers
4k views
Do all interactions terms need their individual terms in regression model?
I am actually reviewing a manuscript where the authors compare 5-6 logit regression models with AIC. However, some of the models have interaction terms without including the individual covariate ...
19
votes
1answer
1k views
How can one empirically demonstrate in R which cross-validation methods the AIC and BIC are equivalent to?
In a question elsewhere on this site, several answers mentioned that the AIC is equivalent to leave-one-out (LOO) cross-validation and that the BIC is equivalent to K-fold cross validation. Is there ...
16
votes
4answers
8k views
What do the residuals in a logistic regression mean?
In answering this question John Christie suggested that the fit of logistic regression models should be assessed by evaluating the residuals. I'm familiar with how to interpret residuals in OLS, they ...
13
votes
4answers
13k views
Negative values for AICc (corrected Akaike Information Criterion)
I have calculated AIC and AICc to compare two general linear mixed models; The AICs are positive with model 1 having a lower AIC than model 2. However, the values for AICc are both negative (model 1 ...
13
votes
2answers
2k views
How should mixed effects models be compared and or validated?
How are (linear) mixed effects models normally compared against each other? I know likelihood ratio tests can be used, but this doesn't work if one model is not a 'subset' of the other correct?
Is ...
11
votes
2answers
2k views
Can AIC compare across different types of model?
I'm using AIC (Akaike's Information Criterion) to compare non-linear models in R. Is it valid to compare the AICs of different types of model? Specifically, I'm comparing a model fitted by glm versus ...
10
votes
2answers
5k views
Understanding AIC and Schwarz criterion
I am running a logistic model. The actual model dataset has more than 100 variables but I am choosing a test data set in which there are around 25 variables. Before that I also made a dataset which ...
10
votes
1answer
196 views
AIC for non-nested models: normalizing constant
The AIC is defined as $AIC=-2 \log(L(\hat\theta))+2p$, where $\hat\theta$ is the maximum likelihood estimator and $p$ is the dimension of the parameter space. For the estimation of $\theta$, one ...
9
votes
2answers
238 views
Problem with comparing GLM models having a different link function
Given the same set of covariates and distribution family, how can I compare models having different link functions?
I think the correct answer here is "AIC/BIC", but I am not 100% sure.
Is it ...
9
votes
3answers
1k views
AIC & BIC number interpretation
I am looking for examples of how to interpret AIC (Akaike information criterion) and BIC (Bayesian information criterion) estimates.
Can negative difference between BICs be interpreted as the ...
8
votes
2answers
774 views
Comparing regression models
I recently fit 4 multiple regression models for the same predictor/response data. Two of the models I fit with Poisson regression.
...
8
votes
1answer
318 views
What is the difference between AIC() and extractAIC() in R?
The R documentation for either does not shed much light. All that I can get from this link is that using either one should be fine. What I do not get is why they are not equal.
Fact: The stepwise ...
7
votes
2answers
2k views
Testing the difference in AIC of two non-nested models
The whole point of AIC or any other information criterion is that less is better. So if I have two models M1: y = a0 + XA + e and M2: y = b0 + ZB + u, and if the AIC of the first (A1) is less than ...
7
votes
1answer
712 views
Is it possible to calculate AIC and BIC for lasso regression models?
Is is possible to calculate an AIC or BIC values for lasso regression models and other regularized models where parameters are only partially entering the equation. How does one determine the degrees ...
6
votes
3answers
9k views
AIC or p-value: which one to choose for model selection?
I'm brand new to this R thing but am unsure which model to select.
I did a stepwise forward regression selecting each variable based on the lowest AIC. I came up with 3 models that I'm unsure which ...
6
votes
1answer
729 views
Why does AIC formula in R appear to use one extra parameter than expected?
I'll use an example so that you can reproduce the results
...
6
votes
1answer
429 views
What is the difference in what AIC and c-statistic (AUC) actually measure for model fit?
Akaike Information Criterion (AIC) and the c-statistic (area under ROC curve) are two measures of model fit for logistic regression. I am having trouble explaining what is going on when the results of ...
6
votes
1answer
118 views
Can you compare AIC values as long as the models are based on the same dataset?
I am doing some forecasting in R using Rob Hyndman's forecast package. The paper belonging to the package can be found here.
In the paper, after explaining the automatic forecasting algorithms, the ...
6
votes
1answer
563 views
Does there exist a model fit statistic (like AIC or BIC) that can be used for absolute instead of just relative comparisons?
I'm not that familiar with this literature, so please forgive me if this is an obvious question.
Since AIC and BIC depend on maximizing the likelihood, it seems that they can only be used to make ...
6
votes
1answer
461 views
Model-selection for linear mixed models over alternative sets of parameters (nlme function in R)
My models look like:
...
5
votes
7answers
582 views
One sentence explanation of the AIC for non-technical types
I need a one-sentence explanation of the use of the AIC in model-building. So far I have "Simply put, the AIC is a relative measure of the amount of observed variation accounted for by different ...
5
votes
2answers
257 views
Why does log likelihood function for a model use SSE/n and not SSE/df?
I'm trying to find out how log-likelihood function works for linear regression. I found the formula here and here. Making some experiments with it (see code below), I was quite surprised that the ...
5
votes
1answer
571 views
BIC or AIC to determine the optimal number of clusters in a scale-free graph?
I am currently trying to partition a scale-free ("big") graph (around 20k vertices, 500k edges) into appropriate sub-graphs. Having derived the Laplacian of the graph, I tried running an approach ...
5
votes
1answer
557 views
Comparing model fits across a set of nonlinear regression models
CONTEXT:
I am modelling the relation between time (1 to 30) and a DV for a set of 60 participants. Each participant has their own time series.
For each participant I am examining the fit of 5 ...
4
votes
2answers
397 views
Why do AIC and BIC show inversed outputs?
I am comparing three relatively simple GLMs having a Gamma distribution with AIC and BIC. The aim is to identify the effects of fertilizers (fdung), year and site on biomass of a specific grass ...
4
votes
2answers
266 views
How to bootstrap the best fit distribution to a sample?
If I have a sample:
set.seed(0)
x <- rlnorm(500)
Then I can use the fit.distr function to find the best fit among two candidate distributions, e.g.
...
4
votes
2answers
786 views
Why applying model selection using AIC gives me non-significant p-values for the variables
I have some questions about the AIC and hope you can help me. I applied model selection (backward, or forward) based on the AIC on my data. And some of the selected variables ended up with a p-values ...
4
votes
1answer
49 views
Equivalence between single sample cross-validation index and the Akaike information criterion for prediction
In "Cross-Validation Methods. Journal of mathematical psychology, Vol. 44, No. 1. (March 2000), pp. 108-132", Professor Browne pointed out that single sample cross-validation index and the Akaike ...
4
votes
2answers
253 views
Generalized Linear Model in SPSS with common values among predictors treated as subpopulations. Why?
I am teaching a class on logistic regression with SPSS. The textbook supplies a sample data set with a binary predictor and two numeric covariates. The sample contains 1000 rows and a number of these ...
4
votes
2answers
165 views
Can AIC determine which data better fit the same model?
Many moons ago, I asked how to differentiate between two very similar non-linear fits and which was better. Finally got that all straightened out after many headaches and three different software ...
4
votes
1answer
464 views
A problem with two ways of calculating AIC
I find that there are two ways to calculate AIC:
AIC = -2ln(likelihood)+ 2K and
AIC = n*ln(RSS/n)+2K
I have:
...
4
votes
1answer
832 views
Why do I get equal AIC, BIC and log likelihood for different models in LME framework?
I have two LME models with the same interaction, one containing both main effects and one containing only one main effect, say :
$$ H\_CE = Season + Crownlevel + Season:Crownlevel , random = ...
4
votes
1answer
195 views
What to choose from BIC/AIC/ridge/elastic net?
I have the following regression problem
I have about 60 independent variables; some of them have a high correlation with others. I have around 3 million observations
(1) - My main goal is ...
4
votes
1answer
434 views
Will AIC and R-square rank models similarly if the number of variables is equal?
This is a fairly basic question: If a number of regression models are being evaluated for fit to a dataset, and each model has 3 variables, will AIC and R-square rank the models similarly? I'm ...
4
votes
2answers
426 views
Relative variable importance with AIC
I am confused and just need some confirmation about calculating the relative variable importance value for the co-variates I used in AIC model selection procedures. I know that there is this one ...
4
votes
0answers
76 views
Is it reasonable to calculate AIC of a subset of the data set which was used to fit the model?
There is a factor variable called "Treatment" in my data set. This factor consists of two levels, "C" and "H". I want to test whether there is there any significant difference between two levels. I ...
3
votes
3answers
3k views
What is AIC? Looking for a formal but intuitive answer
I've heard that AIC can be used to choose among several models (which regressor to use).
But i would like to understand formally what it is in a kind of "advanced undergraduated" level,which i think ...
3
votes
2answers
219 views
Prerequisites for AIC model comparison
What are exactly the prerequisites, that need to be fulfilled for AIC model comparison to work?
I just came around this question when I did comparison like this:
...
3
votes
1answer
449 views
Why is AIC from multinom and corresponding glm different?
I'm reading the example of the book: "S-PLUS (and R) Manual to Accompany Agresti’s Categorical Data Analysis (2002) 2nd edition " (page 55) and when I try to reproduce the example I get this:
...
3
votes
1answer
551 views
Question on AIC and stepAIC
AIC(lm(Fertility ~ ., data=swiss))
[1] 326.0716
ok, since AIC is calculated as
...
3
votes
3answers
666 views
Using AIC, for model selection when both models are equally weighted, and one model has fewer parameters
I am using AIC (Akaike information criterion) for model selection. There are 2 models. The first model has 2 parameters with log likelihood of -10182.0284 and the second model has 3 parameters with ...
3
votes
1answer
232 views
Positive log likelihood values and penalty of more complex models when ranking models using AIC
I am ranking candidate models using Akaike information criterion (AIC).
All my models have positive -2*LL (log likelihood) values which as far as I understand is expected under certain circumstances ...
3
votes
2answers
437 views
Which measure of model fit to report when performing likelihood based regression: AIC, BIC, Pseudo R-square?
I'd like to hear your opinions on the following:
What parameters would you report when estimating different likelihood based regression? AIC, BIC, Pseudo $R^2$?
What is the standard to report?
...
3
votes
1answer
619 views
How to interpret decreasing AIC but higher standard errors in model selection?
I've got a problem choosing the right model. I have a model with various variables (covariables and dummy variables). I was trying to find the best size for this model, so I first started by comparing ...
3
votes
3answers
76 views
Analogous measure of AIC which uses the posterior distribution for model selection?
Suppose the following problem: I have $n$ models, $M_k$, each with parameters $\mathbf{\theta}_k$ for a data set $D$. There where previous observations of a subset of the parameters which are common ...
3
votes
2answers
170 views
Looking for ways to compare between coxph models
I'm running Cox proportional hazards regression in R, and would like to test the option of categorizing one of my continuous variables to factor (I'm aware of the loss of data issue, just checking).
...
3
votes
0answers
148 views
AIC with Mantel's tests
Mantel's tests are commonly used to compare genetic distances (say, between a number of individuals) with true or hypothesized landscape distances between those same individuals. For example, “does ...
3
votes
0answers
331 views
Are very large log likelihood and delta AIC values problematic for model selection?
I am using AICc for small sample sizes to compare 8 a priori models (including null model). I fitted my models using a GLMM due to the nested nature of my data and defined the family as 'poisson' ...
3
votes
0answers
125 views
Compare fits of model to transformed and untransformed response
I want to compare data that proportions among three different groups e.g.:
...
