Questions tagged [information-criteria]

Information-criteria for model selection such as aic, bic, dic, mdl and others

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

Augmented Dickey Fuller test, determining the number of lags used

In a paper by Mark P. Taylor he conducts ADF tests on exchange rates and he describes the determination of lag number with "The number of lagged dependents that we need to include to induce ...
1
vote
1answer
47 views

Information Criteria and Sample size

In the estimation of the univariate time series model, we need to determine the correct order. For the general ARMA (p,q) model, we can determine the true order by information criteria $$AIC(p,q) = ln(...
1
vote
0answers
26 views

Nonparametric estimation and kolmogorov sufficient statistics

Recently, I decided to revisit Cover and Thomas, and yesterday I encountered a very interesting passage in the chapter on Kolmogorov complexity: What does this "different procedure" look ...
1
vote
0answers
52 views

Forecasting: AIC, AICc and BIC VS Cross Validation for model selection (cases for different horizons)

The majority of the automatic model selection algorithms like auto.arima and ets (https://robjhyndman.com/publications/automatic-...
1
vote
1answer
50 views

Does AIC criterion for variable selection in the least square method require the data to be normally distributed?

I'm studying Linear Regression from the book A Modern Approach to Regression with R by Simon J. Sheather (the 2009 edition). The chapter 7 (page 228), in which different criterion for variable ...
0
votes
0answers
42 views

Are information criteria (AIC and SIC) comparable between a linear regression model and a time series model?

I have a problem in which I have to choose the best model between an AR(3) and a MA(3) by comparing the Akaike and Schwarz information criteria. That is fine and easy. The problem comes when I have to ...
0
votes
0answers
24 views

How to use BIC (Bayesian Information Criterion) if the data are not identically distributed but rely on other covariate?

I'm now constructing some models and would like to compare and select the models. I read the wikipedia and some slides about BIC, then I found the ML(maximum likelihood) part in BIC seems to be based ...
2
votes
1answer
66 views

Is there any rationale for rules of thumb for maxlag selection?

I understand that the optimum number of lags can be selected using some information criterion (e.g. Akaike (AIC), Schwartz Bayesian Information Criterion (SBIC) etc.). However, in order to select the ...
0
votes
0answers
47 views

Compare two models over different datasets

The problem: My datasets consist of the brightness of an astronomical object in different frequencies (radio, optical, x-ray etc). I gathered data for several days as I observed the brightness of the ...
1
vote
1answer
193 views

How to get log-likelihood from squared deviance in Scikit Learn

The score() function computes D^2, the percentage of deviance explained, but I'd like to get the log-likelihood to calculate BIC. What's the formula to go from deviance to log-likelihood? Score ...
0
votes
0answers
35 views

More efficient way to find the model with the lowest AIC?

I am working on the zero-inflated Poisson model with a dataset, and trying to find a combination of variables that yields the lowest AIC. I used a series of for loops in R, but it takes forever to get ...
4
votes
1answer
84 views

Model comparison with intractable likelihood using approximate Bayesian Computation

I have some models based on stochastic differential equations (SDEs). Because of the definition of these models, I can simulate data, but I cannot compute the likelihood function / distribution ...
0
votes
0answers
21 views

How to choose information criterion?

In time series analysis, it is often important to determine the optimum number of lags in order to remove serial correlation. For example, in VAR, Dickey-Fuller unit-root test or Granger causality ...
1
vote
0answers
93 views

The similarity between Mallows Cp and AIC?

It is possible to compute the log-likelihood used for AIC as $n /log(RSS/n) + const$ or even as $RSS/\sigma^2 + n\log(\sigma) + const$ considering the least-square or MLE scenario for linear and non-...
2
votes
0answers
19 views

How can be information criteria used with non-independent samples?

I'm puzzled by the following. Many, if not all, of the Information Criteria (AIC, WAIC, LOOIC, PSIS, ...) rely on the independence of the samples -- that we can remove a part, that one point is ...
5
votes
1answer
110 views

The Ambiguity in Schwarz Information Criterion Definition

Suppose there are 100 countries, $i = 1, 2, ..., 100$. Let $b_i$ be the median birth weight of all new born boys in country #i in 2019. Let $g_i$ be the median birth weight of all new born girls in ...
1
vote
1answer
100 views

Differences between formulas for AIC and BIC

I have a question regarding the information criteria AIC and BIC: I found different formulas for the AIC/BIC, the common ones including the likelihood $\mathcal{L}$ are $$AIC = 2K - 2 ln(\mathcal{L})\...
0
votes
1answer
359 views

Log-Likelihood Computation for AIC & BIC

Considering $n$ observations that an be modelled by a Gaussian error model and two nested motion models with $p = 4$ and $p = 7$ parameters, I want to compute the log likelihoods $L$ given the Maximum ...
0
votes
1answer
113 views

Criterion to Pick Number of hidden layers for MLP based on score

I'm training a classifier with a set of training data and checking the result with a set of test data. I'm using sklearn score based on all data and iteratively ...
0
votes
0answers
37 views

How can I compare the performance of model selected by AIC and K-fold cross validation?

guys. I try to compare the model selection criterions. The model is built as a linear regression with 20 explanatory variables. More precisely, what I did so far, is spilting the data into training(70%...
1
vote
2answers
1k views

Feature subset selection by stepwise regression for a random forest model?

I would like to build a random forest model for regression. I have an abundance of potential features, and I expect only some of them to have a significant impact on the target variable. In addition, ...
0
votes
1answer
171 views

Clarification on Akaike's IC (AIC) and BIC for Expectation Maximization with time-changing parameters

I apologize in advance for the trivial question, but I need a clarification on the following issue. Suppose I have a generic model in state-space form described as $$x_{t+1}=\phi_{t} x_{t}+w_{t+1}$$ $...
2
votes
0answers
219 views

WAIC for model comparisons--overly conservative?

I'm having a hard time wrapping my head around the relationship between model posterior predictions and model comparisons via WAIC. Specifically, how do I interpret findings where a model including ...
6
votes
2answers
669 views

Why is the bayesian information criterion called that way?

The word "Bayes" suggests that we are updating a distribution using data, to get a posterior distribution. The fact that the Bayesian information criterion (BIC) is used to select a model from a set ...
0
votes
0answers
35 views

is there a measure for the roughness of a contour plot

There has to be a measure for the difference between "instantaneous" change of "energy" along a line in a space compared to averaged changed of energy along a line. I could take a smooth surface in ...
2
votes
0answers
175 views

Using minimum description length for a categorical distribution

My data is as follows: in a routing node (check figure ), I can see the entry and exit gate of each packet, so I have pairs like this: (1,5) (2,5) (1,6) ... where ...