Questions tagged [information-criteria]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
1answer
39 views

Does AIC criterion for variable selection in the least square method require the data to be normally distributed?

I'm studying Linear Regression from the book A Modern Approach to Regression with R by Simon J. Sheather (the 2009 edition). The chapter 7 (page 228), in which different criterion for variable ...
0
votes
0answers
24 views

Are information criteria (AIC and SIC) comparable between a linear regression model and a time series model?

I have a problem in which I have to choose the best model between an AR(3) and a MA(3) by comparing the Akaike and Schwarz information criteria. That is fine and easy. The problem comes when I have to ...
0
votes
0answers
22 views

How to use BIC (Bayesian Information Criterion) if the data are not identically distributed but rely on other covariate?

I'm now constructing some models and would like to compare and select the models. I read the wikipedia and some slides about BIC, then I found the ML(maximum likelihood) part in BIC seems to be based ...
2
votes
1answer
41 views

Is there any rationale for rules of thumb for maxlag selection?

I understand that the optimum number of lags can be selected using some information criterion (e.g. Akaike (AIC), Schwartz Bayesian Information Criterion (SBIC) etc.). However, in order to select the ...
0
votes
0answers
29 views

Compare two models over different datasets

The problem: My datasets consist of the brightness of an astronomical object in different frequencies (radio, optical, x-ray etc). I gathered data for several days as I observed the brightness of the ...
1
vote
1answer
72 views

How to get log-likelihood from squared deviance in Scikit Learn

The score() function computes D^2, the percentage of deviance explained, but I'd like to get the log-likelihood to calculate BIC. What's the formula to go from deviance to log-likelihood? Score ...
0
votes
0answers
31 views

More efficient way to find the model with the lowest AIC?

I am working on the zero-inflated Poisson model with a dataset, and trying to find a combination of variables that yields the lowest AIC. I used a series of for loops in R, but it takes forever to get ...
3
votes
1answer
67 views

Model comparison with intractable likelihood using approximate Bayesian Computation

I have some models based on stochastic differential equations (SDEs). Because of the definition of these models, I can simulate data, but I cannot compute the likelihood function / distribution ...
0
votes
0answers
19 views

How to choose information criterion?

In time series analysis, it is often important to determine the optimum number of lags in order to remove serial correlation. For example, in VAR, Dickey-Fuller unit-root test or Granger causality ...
1
vote
0answers
83 views

The similarity between Mallows Cp and AIC?

It is possible to compute the log-likelihood used for AIC as $n /log(RSS/n) + const$ or even as $RSS/\sigma^2 + n\log(\sigma) + const$ considering the least-square or MLE scenario for linear and non-...
2
votes
0answers
18 views

How can be information criteria used with non-independent samples?

I'm puzzled by the following. Many, if not all, of the Information Criteria (AIC, WAIC, LOOIC, PSIS, ...) rely on the independence of the samples -- that we can remove a part, that one point is ...
4
votes
1answer
102 views

The Ambiguity in Schwarz Information Criterion Definition

Suppose there are 100 countries, $i = 1, 2, ..., 100$. Let $b_i$ be the median birth weight of all new born boys in country #i in 2019. Let $g_i$ be the median birth weight of all new born girls in ...
1
vote
1answer
86 views

Differences between formulas for AIC and BIC

I have a question regarding the information criteria AIC and BIC: I found different formulas for the AIC/BIC, the common ones including the likelihood $\mathcal{L}$ are $$AIC = 2K - 2 ln(\mathcal{L})\...
0
votes
1answer
239 views

Log-Likelihood Computation for AIC & BIC

Considering $n$ observations that an be modelled by a Gaussian error model and two nested motion models with $p = 4$ and $p = 7$ parameters, I want to compute the log likelihoods $L$ given the Maximum ...
0
votes
1answer
69 views

Criterion to Pick Number of hidden layers for MLP based on score

I'm training a classifier with a set of training data and checking the result with a set of test data. I'm using sklearn score based on all data and iteratively ...
0
votes
0answers
28 views

How can I compare the performance of model selected by AIC and K-fold cross validation?

guys. I try to compare the model selection criterions. The model is built as a linear regression with 20 explanatory variables. More precisely, what I did so far, is spilting the data into training(70%...
1
vote
2answers
927 views

Feature subset selection by stepwise regression for a random forest model?

I would like to build a random forest model for regression. I have an abundance of potential features, and I expect only some of them to have a significant impact on the target variable. In addition, ...
0
votes
1answer
153 views

Clarification on Akaike's IC (AIC) and BIC for Expectation Maximization with time-changing parameters

I apologize in advance for the trivial question, but I need a clarification on the following issue. Suppose I have a generic model in state-space form described as $$x_{t+1}=\phi_{t} x_{t}+w_{t+1}$$ $...
2
votes
0answers
186 views

WAIC for model comparisons--overly conservative?

I'm having a hard time wrapping my head around the relationship between model posterior predictions and model comparisons via WAIC. Specifically, how do I interpret findings where a model including ...
6
votes
2answers
572 views

Why is the bayesian information criterion called that way?

The word "Bayes" suggests that we are updating a distribution using data, to get a posterior distribution. The fact that the Bayesian information criterion (BIC) is used to select a model from a set ...
0
votes
0answers
34 views

is there a measure for the roughness of a contour plot

There has to be a measure for the difference between "instantaneous" change of "energy" along a line in a space compared to averaged changed of energy along a line. I could take a smooth surface in ...
2
votes
0answers
173 views

Using minimum description length for a categorical distribution

My data is as follows: in a routing node (check figure ), I can see the entry and exit gate of each packet, so I have pairs like this: (1,5) (2,5) (1,6) ... where ...