Questions tagged [aic]
AIC stands for the Akaike Information Criterion, which is one technique used to select the best model from a class of models using a penalized likelihood. A smaller AIC implies a better model.
1,018
questions
0
votes
0
answers
23
views
BMA formula with BIC
I am interested in using Bayesian modele averaging as a selection creteria (BMA) vs AIC. I read that BMA is widely implemented in clustering models.
Suppose that we need to fit M models to a data and ...
- 640
0
votes
0
answers
18
views
AIC-BIC in mixed models
I was reading about mixed models and I am confused about AIC and BIC criteria. My first question is can I use this types to calculate them?
AIC=2d-2ln(l)
BIC=dln(n)-2ln(l)
where d: is the numbers of ...
- 21
2
votes
0
answers
15
views
Reporting AIC values for model fits with multiple runs - report the min or the average?
I have a model that I fit with NLL minimization.
I fit that model n times with random starting values, to try to avoid local minima.
When reporting the results, it seems reasonable to me to report the ...
0
votes
0
answers
27
views
Model comparsion for robust linear mixed models (robustlmm)
I'm currently working on a project where I've fitted 4 robust linear mixed models. However, I've hit a bit of a roadblock when it comes to model selection. I've been using the AIC (Akaike Information ...
1
vote
0
answers
33
views
Using step() and car::vif(): order matters?
When fitting linear models and coming up with a plausible one, AIC and VIF are often used. However, I notice that the order in which the methods are used makes a difference on the final model.
Should ...
- 1,433
1
vote
0
answers
18
views
Is it possible average an information criterion across models?
Is it possible to take the average of information criterion like the AIC?
For my model comparison, I have 24 different models. I use 4 different GARCH models each with 6 different distributions for ...
- 11
0
votes
0
answers
31
views
Use linear mixed model or linear quantile mixed model for non-normal residuals?
I started with this initial model:
m1 <- lmer(response ~ treatment + (1|subjectID), data = data)
However, the residuals of the model are heavy-tailed (presumably enough to violate the normality ...
- 63
6
votes
1
answer
279
views
Does information criteria (AIC, BIC and DIC...) imply "causality"?
I am interested in finding out the graphical causal structure. Causal Discovery algorithms (e.g., DAG learning) are used to identify potential causal graphs. In score-based causal discovery methods, ...
- 61
0
votes
0
answers
14
views
How to manually calculate dAIC (survey design-based AIC) in R survey package
In preparation for fitting some more complicated distributions to survey-weighted data, I am trying to reproduce the $d\text{AIC}$ by hand for a simpler distribution (binomial in this case). I am ...
- 1,554
1
vote
0
answers
47
views
Residuals and AIC
AIC is normally calculated using the maximum likelihood, so you must have some probability distribution to work with. But I saw some formulas using the sum of squared residuals or the mean absolute ...
0
votes
0
answers
20
views
What is the correct calculation for AIC corrected (AICc) for a bagged random forest model using the Boston Housing data set? [duplicate]
This question of calculating AIC was answered for a specific linear model here: Calculating AIC “by hand” in R
The problem and solution for a linear model are as follows:
...
1
vote
0
answers
42
views
Which evaluation metric should I choose? AIC or MSE?
I am currently at a total loss, so I hope someone can point me in the right direction regarding my model selection.
The situation
I want to create a linear model that best forecasts my data. I am ...
- 53
0
votes
0
answers
17
views
Should I calculate a value for the AIC based on a test set or a training set?
I am aware that a question very similar to mine has already been asked here (Should AIC be reported on training or test data?), but some points remain unclear to me.
The accepted answer states:
On ...
- 53
0
votes
0
answers
24
views
Rejection of ADF-test for log returns and AIC selected ARIMA(0,0,0) and ARIMA (0,0,0) with a drift?
I use monthly log returns for some stock portfolios and rejects the null of the ADF-test for both. Hereafter I use AIC to select best fitting models using auto.arima in R. The selected models are ...
0
votes
0
answers
51
views
AIC and BIC Manual Calculations Are a Bit Off From Statsmodels Estimates in Python
I ran a multiple regression using statsmodels. I wanted to verify my understanding of calculations for log-likelihood (ll), AIC and BIC. So I attempted to manually calculate the ll, AIC and BIC for ...
- 141
2
votes
0
answers
54
views
Is AIC scale invariant for problems concerning the number of data points in regression?
I am trying to use Akaike Information Criterion with the small sample correction (AICc) as method for determining how many data points to use in a linear approximation of a non-linear function; the ...
- 21
1
vote
0
answers
14
views
General question on model weighting and averaging
I've had a stats related question I've been wondering for a while, but for which I have yet to find good sources on. I'm aware of things like the Akaike Information Criterion for weighting candidate ...
- 11
0
votes
0
answers
47
views
Difference between AICc delta<2 and AICc delta<4 in model.average
I am running a PGLS (Phylogenetic Generalized Least Squares) model selection using the library 'MuMIn' and its functions, see below the code:
...
0
votes
0
answers
9
views
Model Comparison for Overlapping Models (Choosing between the different measurements of the same skill)
I have a large dataset with 15 independent variables. I am interested in investigating the potential interactions between certain independent variables; however, some independent variables measure the ...
- 1
0
votes
0
answers
42
views
If I have T observations from a binomial(n,p), how to consistently estimate n and p?
Suppose I have a dataset $\{S_t\}_{t=1}^T$, where $S_t\overset{i.i.d.}{\sim}Binomial(n,p)$, how to consistently estimate $n$ and $p$ using this dataset? It would be great if you could provide a method ...
- 2,516
0
votes
0
answers
46
views
AIC and Adjusted R squared for fixed number of parameters in sequential selection
Suppose we fix the number of parameters used, why do AIC and adjusted R squared give the same combination of variables?
Observed from the R leaps package that regsubsets gives only one combination of ...
- 37
0
votes
0
answers
27
views
Likelihood of least squares estimates of Vasicek model
I want to compare some short-rate models based on likelihood and AIC. I will use least squares estimates.
Let's take the Vasicek model as an example and its discretized version:
$$
dr = \alpha(\beta-r)...
- 121
1
vote
1
answer
62
views
Results from extractAIC() from a mixed model lmer() are confusing
I have some legacy code in larger block of looped code that compares models.
It essentially boils down to this similar example:
...
- 13
1
vote
1
answer
61
views
Normalising AIC to compensate for missing data?
AIC is simply penalised log-loss, and log-loss depends directly on the dataset size. To create a model from data, missing data need to be excluded first. Assuming missing data are spread across ...
- 8,428
4
votes
1
answer
136
views
Selecting the model by bootstrapping: AIC vs. log-loss?
I'm building a predictive model with potentially multiple predictors. To that end, I try different, nested models, each with one more predictor than the previous one and compare their AICs. The AIC ...
- 8,428
4
votes
1
answer
80
views
Visually comparing two models based on the per-observation AIC
In order to visually compare two models (logistic regression, in case that it matters) I thought of plotting the contribution of the individual observations to the AIC of the respective model. The ...
- 8,428
1
vote
1
answer
59
views
AIC for ARIMA model
While studying univariate time series analysis, I got curious about how I can apply the AIC metric to determine the appropriate order for an ARIMA model.
As far as I've investigated, in many texts ...
- 11
0
votes
0
answers
62
views
step function for lmer models
The step function in R allows me to select a model based on AIC.
lm1 <- lm(mpg ~ ., data = mtcars)
slm1 <- step(lm1)
Is ...
- 1,181
2
votes
1
answer
112
views
Why does using "REML" in mgcv give error for generalized additive models?
I have fitted gam for a generated data with binomial responses. The problem is, many times while running this gam with different bootstrap samples, error occurred.
user369390
1
vote
0
answers
52
views
GLMER and AIC: convergence problems
I am trying to find the "best" model for a binomial GLMER using AIC in R. Here is the equation:
Survival~Age+Group+Age:Group+Age^2+Age^3+(1|Year)+(1|Individual)
Where Year and Individual ...
- 11
1
vote
1
answer
61
views
Is there a function like auto.arima which gives the best VAR model according to a metric like AIC? [closed]
The function auto.arima from the forecast package, automatically fits the best ARIMA model given a time series, by evaluating on a metric like AIC, AICc or BIC.
I ...
2
votes
0
answers
333
views
stepAIC in mixed models
As I haven't found the equivelant of the MASS::stepAIC for mixed models (eg in lmer) what I'm intending to do is to find the best lm model using stepAIC and then go in lmer and add the random effects.
...
- 337
4
votes
3
answers
655
views
Why would a smaller AIC than BIC lead to an increased chance of both overfitting and underfitting?
I am puzzled by the following statement in my lecture notes
AIC penalty smaller than BIC; increased chance of overfitting
BIC penalty bigger than AIC; increased chance of underfitting
Is there a ...
- 641
0
votes
0
answers
46
views
Sample size for comparison of linear regression models
The goal of a study is to test the hypothesis that the inclusion of an additional predictor variable in a multiple linear regression model yields significantly better fit / lower AIC than an otherwise ...
- 43
1
vote
0
answers
50
views
Why/when to set C-hat to the dispersion parameter in Quasi-AIC
In the documentation for MuMIn::QIAC, and several other sources on calculating QAIC, the following code snippet appears which I don't understand at all.
...
- 111
0
votes
0
answers
16
views
Comparing two different Regression Models using the same set of Independent Variables
I'm currently working on the same set of independent variables to explain FDI across different time series data; hence, I'll like to know if there is a statistical tool that tells which of the model ...
- 1
1
vote
1
answer
86
views
How to pick a model based on AIC and number of factors?
I'm trying to create a linear mixed model for some of my data. I'm brute forcing a backwards step selection by taking out the least significant parameter both times. I went back 6 fixed effects and ...
- 145
0
votes
0
answers
21
views
Selecting a forecasting model with AIC: some questions
I'm using maximum likelihood estimation to fit a model to time series data. I later want to use the model to do forecasting. Let $L$ be the likelihood function,
$$L_i = L(f_k(\theta), data_i).$$
It ...
- 21
1
vote
0
answers
23
views
How do you compare the strength of predictor variables?
We have been assessing the prognostic utility of MRI markers in a patient population. I identified markers-of-interest based on Cox regression analysis. Each marker had aHR of 5.5, 3.7, 2.8, etc, ...
- 11
2
votes
0
answers
22
views
Aikaike Information Criterion for model with fixed parameter
I am trying to fit an inhomogenious, reinforced Poisson process to time series data using maximum likelihood estimation. The inhomogenious rate is
$$ \lambda = \alpha \cdot f(t,\theta) \cdot (m +n).$$
...
- 21
0
votes
0
answers
53
views
Do AIC improvements make multicollinearity 'worth it'? (and an analagous question about GAM concurvity and fREML)
To what extent is it the case that I can ignore multicollinearity if adding in those terms improves my AIC?
Frankly I'm a little rusty on my theory, but I've seen some people suggesting that AIC is a ...
Tyrone Sawyer
0
votes
0
answers
95
views
Justification for using information criteria (AIC, AICc, BIC) in ARIMA time series model selection
Intro
As far as I understand, AIC, AICc, and BIC information criteria can be used when different models $M_j$ that differ in the number of estimated parameters $K_j$, which are applied to the same $\...
- 301
0
votes
1
answer
2k
views
Johansen Cointegration test in Python (statsmodels)
I have three time series df['a'], df['b'] and df['c'] which I want to test for cointegration ...
- 1
3
votes
1
answer
37
views
Model section when AIC and logLik differ
To analyze my ordinal (Liker scale ratings) I used the clmm function for the ordinal package. Since I have fixed and random ...
- 33
0
votes
1
answer
57
views
I'm getting bigger values of AIC and BIC after applying the p and q (obtained from auto.arima() function) to the standard GARCH model
I tried to find p and q by using auto.arima function in RStudio. It gave ARIMA(1,1,0). However, after applying p=1 and q=0 to the GARCH model, ar1 became ...
- 1
0
votes
0
answers
31
views
Comparing AIC between moderation and mediation models
I am doing a study that was primarily testing two potential moderators (with the lm function in R) and then followed up with exploratory analyses that tested the two as potential mediators (which I ...
- 335
1
vote
0
answers
23
views
Information criterion for cases where $n\gg k$?
I'm looking for an information criterion metric that effectively penalizes the number of parameters ($k$) in cases where we have huge sample sizes. In my particular case my sample size is $\approx80,...
- 123
3
votes
1
answer
337
views
Why is BIC considered consistent (though AIC is mostly used) for large number of observation?
AIC, BIC are the famous criteria for model selection. But many times they show different results.
I read in several places that BIC is consistent while AIC is not. And AIC can achieve minimax rate but ...
user369390
0
votes
0
answers
43
views
To the likelihood of what does the loglikelihood in AIC, BIC refer?
Both Akaike's information criterion and Bayesian information criterion are calculated from the loglikelihood, eg AIC= -2 log-likelihood +2k.
To the likelihood of what does it refer to? The likelihood ...
1
vote
2
answers
60
views
What AIC is necessary to select this model?
I have read this page but am a little confused and I think a real example might help solidify the idea in my mind regarding how to use the AIC in model selection.
Equivalence of AIC and p-values in ...
- 515