Linked Questions
48 questions linked to/from Is there any reason to prefer the AIC or BIC over the other?
7
votes
2
answers
18k
views
When do you use AIC vs. BIC [duplicate]
How do you know when to use AIC or BIC for determining model fit? Is it just a judgment call? Is there an intuitive explanation as to which heuristic is better than the other?
- 1,381
7
votes
1
answer
3k
views
Use BIC or AIC as approximation for Bayesian Model Averaging [duplicate]
I want to compare "real" Bayesian Model Averaging (BMA) performed with the EM algorithm and information-criterion based BMA.
Which one, BIC or AIC, is a "closer" approximation to the "real" BMA?
BIC ...
- 149
1
vote
0
answers
802
views
Interpretation of AIC, BIC and KIC [duplicate]
Possible Duplicate:
Is there any reason to prefer the AIC or BIC over the other?
Can anyone please interpret each term in AIC, BIC and KIC. And the difference between the three.
Thanks in ...
- 11
0
votes
0
answers
30
views
Optimal number of components in a beta mixture model [duplicate]
This is a well-written blog on how we can fit a mixture of beta distributions to a dataset:
http://varianceexplained.org/r/mixture-models-baseball/
However, it would have been excellent to identify ...
- 173
117
votes
12
answers
70k
views
When should linear regression be called "machine learning"?
In a recent colloquium, the speaker's abstract claimed they were using machine learning. During the talk, the only thing related to machine learning was that they perform linear regression on their ...
- 1,527
51
votes
5
answers
227k
views
AIC guidelines in model selection
I typically use BIC as my understanding is that it values parsimony more strongly than does AIC. However, I have decided to use a more comprehensive approach now and would like to use AIC as well. I ...
- 989
11
votes
5
answers
7k
views
Is a variable significant in a linear regression model?
I've got a linear regression model with the sample and variable observations and I want to know:
Whether a specific variable is significant enough to remain included in the model.
Whether another ...
- 780
23
votes
2
answers
10k
views
Why is best subset selection not favored in comparison to lasso?
I'm reading about best subset selection in the Elements of statistical learning book.
If I have 3 predictors $x_1,x_2,x_3$, I create $2^3=8$ subsets:
Subset with no predictors
subset with predictor $...
- 857
25
votes
4
answers
864
views
Addressing model uncertainty
I was wondering how the Bayesians in the CrossValidated community view the problem of model uncertainty and how they prefer to deal with it? I will try to pose my question in two parts:
How important ...
- 3,617
26
votes
2
answers
11k
views
Best approach for model selection Bayesian or cross-validation?
When trying to select among various models or the number of features to include for, say prediction I can think of two approaches.
Split the data into training and test sets. Better still, use ...
- 2,402
33
votes
1
answer
22k
views
Proof of LOOCV formula
From An Introduction to Statistical Learning by James et al., the leave-one-out cross-validation (LOOCV) estimate is defined by $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\text{MSE}_i$$
...
- 5,147
35
votes
4
answers
8k
views
AIC versus cross validation in time series: the small sample case
I am interested in model selection in a time series setting. For concreteness, suppose I want to select an ARMA model from a pool of ARMA models with different lag orders. The ultimate intent is ...
- 69.5k
13
votes
4
answers
66k
views
Interpretation of AIC value
Typical values of AIC that I have seen for logistic models are in thousands, at least hundreds.
e.g. On http://www.r-bloggers.com/how-to-perform-a-logistic-regression-in-r/
the AIC is 727.39
While ...
- 846
14
votes
4
answers
15k
views
Are models identified by auto.arima() parsimonious?
I have been trying to learn and apply ARIMA models. I have been reading an excellent text on ARIMA by Pankratz - Forecasting with Univariate Box - Jenkins Models: Concepts and Cases. In the text the ...
- 8,655
11
votes
3
answers
13k
views
What is AIC? Looking for a formal but intuitive answer
I've heard that AIC can be used to choose among several models (which regressor to use).
But i would like to understand formally what it is in a kind of "advanced undergraduated" level, which I ...
- 119
15
votes
3
answers
3k
views
Are there circumstances in which BIC is useful and AIC is not?
In the Wikipedia entry for Akaike information criterion, we read under Comparison with BIC (Bayesian information criterion) that
...AIC/AICc has theoretical advantages over BIC...AIC/AICc is derived ...
- 13.3k
9
votes
2
answers
3k
views
If the AIC and the BIC are asymptotically equivalent to cross validation, is it possible to dispense with a test set when using them?
Several sources I've come across state that the AIC and the BIC are asymptotically equivalent to cross-validation (see multiple answers here for example, and here), .
When training a predictive ...
- 12.1k
14
votes
2
answers
9k
views
Cross Validation for mixed models?
My colleague and I are fitting a range of linear and nonlinear mixed effect models in R. We are asked to perform cross-validation on the fitted models so that one can verify that the effects observed ...
- 253
23
votes
1
answer
2k
views
Does BIC try to find a true model?
This question is a follow-up or attempt to clear up possible confusion regarding a topic I and many others find a bit difficult, regarding the difference between AIC and BIC. In a very nice answer by @...
- 1,824
15
votes
2
answers
806
views
When to stop refining a model?
I have been studying statistics from many books for the last 3 years, and thanks to this site I learned a lot. Nevertheless one fundamental question still remains unanswered for me. It may have a very ...
- 4,504
18
votes
1
answer
6k
views
Variable selection vs Model selection
So I understand that variable selection is a part of model selection. But what exactly does model selection consist of? Is it more than the following:
1) choose a distribution for your model
2) ...
- 1,824
1
vote
2
answers
7k
views
R - Model selection in Glmer
Having troubles to perform a model selection for glmer in R. I'm using the package lme4 with the following structure:
...
3
votes
2
answers
2k
views
Model Evaluation for Discrete Regression
I've building a model to predict count variables, i. e. the quantity I'm predicting is a positive integer.
I know that for regression a usual metric of model quality is the R-squared coefficient, but ...
2
votes
1
answer
4k
views
AIC, BIC, DIC, model selection criteria
I am trying to understand the difference between these parameters, and their application. Was hoping to get some correction/clarification to my statements. I have a training set and cross-validation ...
- 595
7
votes
2
answers
541
views
Developing a statistical test to ascertain better "fit"
In a data set with thousands of data points, I am testing different short-term and longer term data outputs based on 5 rolling data points all the way to 100 rolling data points (which each value ...
- 189
5
votes
2
answers
1k
views
Which measure of model fit to report when performing likelihood based regression: AIC, BIC, Pseudo R-square?
I'd like to hear your opinions on the following:
What parameters would you report when estimating different likelihood based regression? AIC, BIC, Pseudo $R^2$?
What is the standard to report?
It ...
- 6,023
4
votes
1
answer
1k
views
Prediction vs. Explanation and its Effect on Statistical Methods [duplicate]
In layman's terms, what is the difference between predicting and explaining in statistics? I was looking for the differences between AIC and BIC and found this post with an answer stating:
My quick ...
- 741
2
votes
2
answers
876
views
Distribution comparison by AIC
I'd like to compare several distributions fitted to one dataset (of i.i.d. random variables) by AIC. Do there exist some specific rules of thumb for such a situation?
It seems that most of such rules ...
- 231
1
vote
2
answers
2k
views
What criteria tell us that the prediction of a model is reliable
What criteria can be used to tell whether the prediction of a model will be more reliable than other specifications.
Background:
We have data with $N$ computers.
However, prices available only for, ...
- 153
7
votes
1
answer
527
views
Equivalence between single sample cross-validation index and the Akaike information criterion for prediction
In "Cross-Validation Methods. Journal of mathematical psychology, Vol. 44, No. 1. (March 2000), pp. 108-132", Professor Browne pointed out that single sample cross-validation index and the Akaike ...
- 1,626