AIC stands for the Akaike Information Criterion, which is one technique used to select the best model from a class of models using a penalized likelihood. A smaller AIC implies a better model.
0
votes
0answers
29 views
How do we get log likelihood of KNN?
For getting log-likelihood values, I am using R AIC() method. Although I can get the log-likelihood values of linear regression models, getting the following error for when I applied R AIC() method on ...
1
vote
1answer
18 views
comparing AIC (or BIC or whatever) between different SETS of models
Suppose I have $m$ competing models. Suppose also that I could classify these models into $s$ sets. For example, I could classify models of migration behavior conditional on climatic conditions by the ...
0
votes
0answers
55 views
Selection of lme models using AIC & appropriate random effects & variance structure
I am using three categorical predictor variables X1, X2, X3 and one continuous dependent variable Y, and I want to treat X3 as a random effect.
The simplest model I could come with:
...
1
vote
2answers
92 views
Overfitting when using corrected AIC for model selection
I am using the corrected AIC to select the lag order in a simple AR(p) model. I chose the the AICc since my sample is fairly small (n=135). The AICc minimal model is the AR(15). To me it seems like an ...
0
votes
0answers
53 views
AIC vs BIC vs MDL
I am trying to learn the difference between the three approaches and their applications.
a) As I understand,
AIC = -LL+K
BIC = -LL+(K*logN)/2
Unless I am ...
6
votes
1answer
133 views
Can you compare AIC values as long as the models are based on the same dataset?
I am doing some forecasting in R using Rob Hyndman's forecast package. The paper belonging to the package can be found here.
In the paper, after explaining the automatic forecasting algorithms, the ...
2
votes
2answers
51 views
AIC, BIC parsimony
I've set up code to give me a graphical depiction of AIC vs BIC parsimony over various degrees of polynomial models. On the rare occassion AIC does not match BIC trends, which parsimonious model ...
0
votes
0answers
40 views
lm to lmer function tweaking
I have stolen and modified a snippet of code found off the internet from (http://www.r-bloggers.com/aic-bic-vs-crossvalidation/) which graphically depicts AIC and BIC values for different polynomial ...
0
votes
0answers
25 views
How to calculate the weighted sum of absolute deviations to determine AIC for quantile regression
I would like to know if there is a way to calculate the sum of the weighted absolute deviations for quantile regressions with package quantreg?
I'm following the ...
1
vote
1answer
58 views
Comparing GLS models with different fixed variables using AIC: REML or ML?
I am using gls in nlme. My response variable is spatial so I am using gls with correlation structure. I am determining which structure to use based on Zuur 2009, comparing AIC scores of models with ...
0
votes
0answers
27 views
incorporating averaging models from AIC and still using k-fold cross validation?
Ive a county/district that Ive divided into ~300 grids that are 15km^2 in size attributed with various habitat and economic variables that have been summarized and standardized. I then have 2 types ...
4
votes
1answer
50 views
Equivalence between single sample cross-validation index and the Akaike information criterion for prediction
In "Cross-Validation Methods. Journal of mathematical psychology, Vol. 44, No. 1. (March 2000), pp. 108-132", Professor Browne pointed out that single sample cross-validation index and the Akaike ...
1
vote
0answers
27 views
Is there a way to correct standard errors and/or prediction intervals for multiple comparison after doing backwards selection?
It is well known that most model selection algorithms can easily fall into a multiple comparison trap. To quote Friedman:
Consider developing a regression model in a context where substantive
...
2
votes
2answers
70 views
How do I setup a model with hierarchical structure using lmer in R?
I am trying to isolate the important predictors for my response variable "Y". I know that "TL" (which is an individual level predictor) affects "Y", and now I want to determine if adding the site ...
0
votes
1answer
88 views
AIC, BIC, DIC, model selection criteria
I am trying to understand the difference between these parameters, and their application. Was hoping to get some correction/clarification to my statements. I have a training set and cross-validation ...
2
votes
1answer
70 views
What is the dimension (k) of these regression models?
I am attempting to use Akaike's Information Criterion to select the most appropriate model for some data. This means I need to find the likelihood of my data under various models, compute the AIC ...
3
votes
3answers
77 views
Analogous measure of AIC which uses the posterior distribution for model selection?
Suppose the following problem: I have $n$ models, $M_k$, each with parameters $\mathbf{\theta}_k$ for a data set $D$. There where previous observations of a subset of the parameters which are common ...
1
vote
1answer
46 views
Is it valid to compare AICc among models with different numbers of independent vbls?
I am trying to interpret some results in a paper that presents AICc values for different candidate multiple regression models. The paper presents the model results broken down by numbers of ...
1
vote
0answers
38 views
How do I know if the differences in ICs among candidate models are significant?
I'm doing some exploratory modelling on a data set with 29 covariates and an additional 11 variables that are of interest to my research question.
My strategy is to develop a model with a subset of ...
6
votes
1answer
462 views
What is the difference in what AIC and c-statistic (AUC) actually measure for model fit?
Akaike Information Criterion (AIC) and the c-statistic (area under ROC curve) are two measures of model fit for logistic regression. I am having trouble explaining what is going on when the results of ...
0
votes
0answers
61 views
How do I explain that software implemented model selection procedures should not be used unsupervised?
I know that people generally say that procedures which select a model based on information criterion lead to inconsistent model selections.
I read a paper by Leeb and Potscher (2005), MODEL SELECTION ...
4
votes
2answers
262 views
Generalized Linear Model in SPSS with common values among predictors treated as subpopulations. Why?
I am teaching a class on logistic regression with SPSS. The textbook supplies a sample data set with a binary predictor and two numeric covariates. The sample contains 1000 rows and a number of these ...
1
vote
1answer
190 views
Negative and positive AICc/BIC for two models with transformed data - how to compare?
I am using AICc for model selection for transformed data (continous variable). One model I used $\log_{10}(PWV)$ as response and the other $\log(PWV)$ as response, but I am not sure which one to use ...
2
votes
0answers
55 views
AIC/BIC: how many parameters does a permutation count for?
Let's say I have a model selection problem and I am trying to use AIC or BIC to evaluate the models. This is straightforward for models that have some number $k$ of real-valued parameters.
However, ...
3
votes
2answers
228 views
Prerequisites for AIC model comparison
What are exactly the prerequisites, that need to be fulfilled for AIC model comparison to work?
I just came around this question when I did comparison like this:
...
2
votes
0answers
116 views
LASSO vs AIC for feature selection with the Cox model
I have some questions about the Lasso.
After using the AIC or BIC to select a model, the model is fit with the variables selected in order to get the standard errors of the estimates with CIs, ...
1
vote
1answer
505 views
auto.arima from Forecast package
I am trying to fit a time series using the function auto.arima and I face some strange results.
As a first try, I use the command
...
9
votes
2answers
242 views
Problem with comparing GLM models having a different link function
Given the same set of covariates and distribution family, how can I compare models having different link functions?
I think the correct answer here is "AIC/BIC", but I am not 100% sure.
Is it ...
4
votes
2answers
434 views
Relative variable importance with AIC
I am confused and just need some confirmation about calculating the relative variable importance value for the co-variates I used in AIC model selection procedures. I know that there is this one ...
4
votes
2answers
168 views
Can AIC determine which data better fit the same model?
Many moons ago, I asked how to differentiate between two very similar non-linear fits and which was better. Finally got that all straightened out after many headaches and three different software ...
4
votes
0answers
76 views
Is it reasonable to calculate AIC of a subset of the data set which was used to fit the model?
There is a factor variable called "Treatment" in my data set. This factor consists of two levels, "C" and "H". I want to test whether there is there any significant difference between two levels. I ...
0
votes
0answers
153 views
Can dredge() in R package MuMIn deal with global model objects generated by gls() in nlme?
I am trying to use the function dredge() in the package MuMIn to compare AIC model-selection statistics for models of all ...
0
votes
1answer
26 views
What is the global model?
I'm trying to calculate c-hat, the overdispersion parameter for a QAIC model set. According to Burnham and Anderson, you're supposed to calculate c-hat on the global model. Is the global model the ...
0
votes
0answers
56 views
AIC with multiple linear slopes
I have a set of data points. I would like to know which model of linear slopes best fits the data: two slopes, three slopes, or four slopes. How can I calculate the AIC for each of these models (i.e, ...
4
votes
1answer
195 views
What to choose from BIC/AIC/ridge/elastic net?
I have the following regression problem
I have about 60 independent variables; some of them have a high correlation with others. I have around 3 million observations
(1) - My main goal is ...
8
votes
1answer
325 views
What is the difference between AIC() and extractAIC() in R?
The R documentation for either does not shed much light. All that I can get from this link is that using either one should be fine. What I do not get is why they are not equal.
Fact: The stepwise ...
1
vote
2answers
960 views
REML vs ML stepAIC
I feel overwhelmed after attempting to dig into the literature on how to run my mixed model analysis following it up with using AIC to select the best model or models. I do not think my data is that ...
5
votes
2answers
259 views
Why does log likelihood function for a model use SSE/n and not SSE/df?
I'm trying to find out how log-likelihood function works for linear regression. I found the formula here and here. Making some experiments with it (see code below), I was quite surprised that the ...
0
votes
2answers
134 views
Selecting the best indicator of disease progression
Quite a while ago, I asked a question for which Peter Ellis provided a very interesting answer. Now I'd like to follow on that and have your comments and ideas on how to actually put it to use.
I try ...
3
votes
2answers
173 views
Looking for ways to compare between coxph models
I'm running Cox proportional hazards regression in R, and would like to test the option of categorizing one of my continuous variables to factor (I'm aware of the loss of data issue, just checking).
...
1
vote
1answer
217 views
How to calculate absolute fit indices (RMSEA, GFI…) from relative ones (AIC, BIC…)?
I have conducted an IRT analysis with Conquest in order to compare two models (1-dimensional vs. 8-dimensional) applied to a given data set (41 items of a questionnaire, N=195).
Comparing the ...
2
votes
0answers
191 views
Relation between reduced chi square and Akaike criterion
In choosing the best fit between different curves, supposing the errors are distributed normally, I can do the following things:
Calculate the reduced $\chi_r^2=\frac{\chi^2}{N_b-N_p}$, where $N_b$ ...
10
votes
1answer
196 views
AIC for non-nested models: normalizing constant
The AIC is defined as $AIC=-2 \log(L(\hat\theta))+2p$, where $\hat\theta$ is the maximum likelihood estimator and $p$ is the dimension of the parameter space. For the estimation of $\theta$, one ...
4
votes
2answers
268 views
How to bootstrap the best fit distribution to a sample?
If I have a sample:
set.seed(0)
x <- rlnorm(500)
Then I can use the fit.distr function to find the best fit among two candidate distributions, e.g.
...
4
votes
2answers
794 views
Why applying model selection using AIC gives me non-significant p-values for the variables
I have some questions about the AIC and hope you can help me. I applied model selection (backward, or forward) based on the AIC on my data. And some of the selected variables ended up with a p-values ...
0
votes
1answer
200 views
Good model vs. AIC
Suppose I run a bidirectional stepwise in R with the model:
step(glm(y ~ a + b + c + d, poisson))
And the result may be:
...
9
votes
3answers
1k views
AIC & BIC number interpretation
I am looking for examples of how to interpret AIC (Akaike information criterion) and BIC (Bayesian information criterion) estimates.
Can negative difference between BICs be interpreted as the ...
1
vote
0answers
59 views
Is the AIC/BIC different for different Granger causality directions?
If I have two possible Granger causality setups:
$A_t = intercept + \sum_{i=1}^p a_i L^i (A_t) + \sum_{i=1}^p b_i L^i (B_t) + e_t$
$B_t = intercept+\sum_{i=1}^p a_i L^i (A_t) + \sum_{i=1}^p b_i L^i ...
3
votes
3answers
671 views
Using AIC, for model selection when both models are equally weighted, and one model has fewer parameters
I am using AIC (Akaike information criterion) for model selection. There are 2 models. The first model has 2 parameters with log likelihood of -10182.0284 and the second model has 3 parameters with ...
1
vote
0answers
110 views
Using AIC to distinguish between models using multiple datasets
I want to use AIC to compare three candidate models (labeled by m), each having K_m parameters. However, I have M datasets over which I can make the comparison. My ultimate goal is to report the ...
