Questions tagged [information-criteria]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

How to use BIC (Bayesian Information Criterion) if the data are not identically distributed but rely on other covariate?

I'm now constructing some models and would like to compare and select the models. I read the wikipedia and some slides about BIC, then I found the ML(maximum likelihood) part in BIC seems to be based ...
2
votes
1answer
20 views

Is there any rationale for rules of thumb for maxlag selection?

I understand that the optimum number of lags can be selected using some information criterion (e.g. Akaike (AIC), Schwartz Bayesian Information Criterion (SBIC) etc.). However, in order to select the ...
0
votes
0answers
6 views

Compare two models over different datasets

The problem: My datasets consist of the brightness of an astronomical object in different frequencies (radio, optical, x-ray etc). I gathered data for several days as I observed the brightness of the ...
1
vote
1answer
22 views

How to get log-likelihood from squared deviance in Scikit Learn

The score() function computes D^2, the percentage of deviance explained, but I'd like to get the log-likelihood to calculate BIC. What's the formula to go from deviance to log-likelihood? Score ...
0
votes
0answers
29 views

More efficient way to find the model with the lowest AIC?

I am working on the zero-inflated Poisson model with a dataset, and trying to find a combination of variables that yields the lowest AIC. I used a series of for loops in R, but it takes forever to get ...
3
votes
1answer
40 views

Model comparison with intractable likelihood using approximate Bayesian Computation

I have some models based on stochastic differential equations (SDEs). Because of the definition of these models, I can simulate data, but I cannot compute the likelihood function / distribution ...
0
votes
0answers
14 views

How to choose information criterion?

In time series analysis, it is often important to determine the optimum number of lags in order to remove serial correlation. For example, in VAR, Dickey-Fuller unit-root test or Granger causality ...
1
vote
0answers
46 views

The similarity between Mallows Cp and AIC?

It is possible to compute the log-likelihood used for AIC as $n /log(RSS/n) + const$ or even as $RSS/\sigma^2 + n\log(\sigma) + const$ considering the least-square or MLE scenario for linear and non-...
2
votes
0answers
18 views

How can be information criteria used with non-independent samples?

I'm puzzled by the following. Many, if not all, of the Information Criteria (AIC, WAIC, LOOIC, PSIS, ...) rely on the independence of the samples -- that we can remove a part, that one point is ...
4
votes
1answer
99 views

The Ambiguity in Schwarz Information Criterion Definition

Suppose there are 100 countries, $i = 1, 2, ..., 100$. Let $b_i$ be the median birth weight of all new born boys in country #i in 2019. Let $g_i$ be the median birth weight of all new born girls in ...
1
vote
1answer
76 views

Differences between formulas for AIC and BIC

I have a question regarding the information criteria AIC and BIC: I found different formulas for the AIC/BIC, the common ones including the likelihood $\mathcal{L}$ are $$AIC = 2K - 2 ln(\mathcal{L})\...
0
votes
1answer
91 views

Log-Likelihood Computation for AIC & BIC

Considering $n$ observations that an be modelled by a Gaussian error model and two nested motion models with $p = 4$ and $p = 7$ parameters, I want to compute the log likelihoods $L$ given the Maximum ...
0
votes
1answer
33 views

Criterion to Pick Number of hidden layers for MLP based on score

I'm training a classifier with a set of training data and checking the result with a set of test data. I'm using sklearn score based on all data and iteratively ...
0
votes
0answers
75 views

How to choose optimal lag length VAR Model AIC SC don't work

Good afternoon, I'm currently confronted with a quagmire. I have weekly data (around 500 observations) for two economics time series and I want to find the optimal lag length for a VAR Model with ...
0
votes
0answers
21 views

How can I compare the performance of model selected by AIC and K-fold cross validation?

guys. I try to compare the model selection criterions. The model is built as a linear regression with 20 explanatory variables. More precisely, what I did so far, is spilting the data into training(70%...
0
votes
0answers
21 views

Information Criterion vs. Cross Validation for Time Series

I've seen a few posts regarding this question but was hoping for a thorough explanation and possibly a source to go with it. Why would we prefer IC methods over cross-validation for time series data? ...
1
vote
2answers
482 views

Feature subset selection by stepwise regression for a random forest model?

I would like to build a random forest model for regression. I have an abundance of potential features, and I expect only some of them to have a significant impact on the target variable. In addition, ...
0
votes
1answer
106 views

Clarification on Akaike's IC (AIC) and BIC for Expectation Maximization with time-changing parameters

I apologize in advance for the trivial question, but I need a clarification on the following issue. Suppose I have a generic model in state-space form described as $$x_{t+1}=\phi_{t} x_{t}+w_{t+1}$$ $...
2
votes
0answers
148 views

WAIC for model comparisons--overly conservative?

I'm having a hard time wrapping my head around the relationship between model posterior predictions and model comparisons via WAIC. Specifically, how do I interpret findings where a model including ...
6
votes
2answers
425 views

Why is the bayesian information criterion called that way?

The word "Bayes" suggests that we are updating a distribution using data, to get a posterior distribution. The fact that the Bayesian information criterion (BIC) is used to select a model from a set ...
0
votes
0answers
29 views

is there a measure for the roughness of a contour plot

There has to be a measure for the difference between "instantaneous" change of "energy" along a line in a space compared to averaged changed of energy along a line. I could take a smooth surface in ...