Questions tagged [bic]
BIC is an acronym for Bayesian Information Criterion. BIC is one method of model comparison. See also AIC
311
questions
1
vote
1
answer
90
views
How to use priors on the parameter number with an information criterion (AIC, BIC, …)?
Example
The example is made up because I hope that it’s more accessible than my actual problem.
I want to determine the number of planets of a star.
I have:
data for some astronomical observable of ...
- 2,257
2
votes
0
answers
42
views
Is it circular reasoning to compute the ELBO using MCMC?
Let's say we have a posterior distribution $q(\theta) = p(\theta \mid D, \mathcal{M})$ over parameters $\theta$ given data $D$ and a model $\mathcal{M}$. As is often the case, computing $q$ is hard, ...
- 21
0
votes
0
answers
19
views
Difference between Bayesian Information Criteria and Approximate Bayesian Computation as model selection
My question is not very technical and more like a discussion but I will be happy to have a technical input for the comparison b/w BIC and ABC.
I am trying to understand and use the best model ...
- 141
3
votes
1
answer
53
views
Model selection criteria that represent a compromise between AIC and BIC
I am very familiar with the ideas and formula of the two popular model selection criteria AIC/AICc and BIC. When I use them for practical problems in chemometrics, the use of AIC/AICc often gives ...
- 33
0
votes
0
answers
66
views
SEM model comparison
I have problem choosing a better measurement model between two (i.e., A and B). Both models have acceptable fit indices. For example, the RMSEA of A is .056; the RMSEA of B is .067. By RMSEA, A seems ...
0
votes
1
answer
68
views
AIC and BIC in GEE
Can I apply Akaike’s information criterion (AIC) and Quasi-likelihood under the independence model information criterion (QIC) in GEE?
2
votes
0
answers
33
views
AIC vs BIC for time series clustering and descriptive purposes
I'm in the process of fitting a hidden markov model with gaussian mixtures to time series health data. The primary purpose of this is descriptive, not predictive – I'm using the fitted model to give a ...
- 21
2
votes
2
answers
598
views
AIC, BIC and log likelihood which more important?
I am currently searching for the best ARMA(p,q) model for my conditional mean.
When comparing the AIC, BIC and LL, I see that some model perform better in AIC, some in BIC and some in LL. The AC and ...
- 41
0
votes
0
answers
48
views
Model selection and validation of latent class models with sampling weights
We are conducting a LCA with sampling weights, using Stata. Since we apply these sampling weights, the dataset becomes very large and the goodness of fit tests do not validate the model and do not ...
- 1
1
vote
0
answers
48
views
Distribution of Bayesian Information Criteria - using BIC when there are multiple datasets
I have two competing nested models : $M1$ and $M2$.
$M1$ has way less parameters.
I know that $M2$ is definitely a true model for data (i.e., can explain data fully) but I claim that it is not the ...
- 31
0
votes
0
answers
32
views
Using AIC/BIC to compare models with and without mediators
Our team ran into an issue where we had some confusion as to whether it's appropriate to use AIC or BIC to compare sets of models with and without mediators. That is, in our first model(s) we only ...
- 1
2
votes
1
answer
206
views
How do you write the AIC and BIC of a regression model in terms of the coefficient-of-determination?
This question is to give a general exposition of the relationship between goodness-of-fit statistics in regression analysis, to answer questions like this one.
Consider a nonlinear Gaussian regression ...
- 102k
2
votes
1
answer
136
views
Linear regression gives a good $R^2$ but also a high BIC
I tried a linear model with interactions on my data ($n=95,840$ rows):
...
- 131
10
votes
4
answers
1k
views
In what applications do we prefer Model Selection over Model Averaging?
I'm wondering in what applications or scenarios (or in trying to answer what kind of questions), the researcher would prefer using Model Selection (such as AIC or BIC) over Model Averaging (such as ...
- 1,898
0
votes
0
answers
58
views
What is the number of free parameters in an n-component GMM?
I am trying to calculate BIC = -2logL + log(N)d where d is the number of free parameters or degrees of freedom.
If I am fitting guassian mixture model to the data, ...
- 1
0
votes
1
answer
45
views
Regression curve with lowest combined AIC and BIC is a poor predictor
For fun, I was trying to make a predictor for how long it would take for George R. R. Martin's The Winds of Winter to be released. My "best" model is the one that had the lowest combined AIC ...
- 121
0
votes
0
answers
32
views
Should I use AIC / BIC or rather cross validation for discovering gov. equations through linear regression (SINDy)?
I want to use linear regression with very large design matrix for discovery of governing equations to i.e. physical systems. The design matrix would include potential terms that can be part of the ...
- 11
5
votes
2
answers
219
views
How to calculate the degrees of freedom for L1 and L2 regularised GLMs?
My goal is to calculate various information criteria for generalised linear models (e.g., the AIC). To do this, we need to calculate the effective degrees of freedom of the trained model. In an ...
- 151
0
votes
0
answers
31
views
ODE model selection criterion selection criteria
There is substantial literature on model selection criteria and many questions around this topic on CrossValidated. However, I could not find one that covers my case.
I have a time series dataset (of ...
- 11
5
votes
0
answers
84
views
Brief characterizations of AIC and BIC: how helpful are they?
I have found the following one-sentence characterizations of AIC and BIC in a lecture note:
AIC estimates the degree to which the predictive accuracy of the model will generalize to new data.
BIC ...
- 57.5k
1
vote
1
answer
46
views
Interpreting why a VAR produces lower error than VARMA?
I trained various VARMA models on the same dataset consisting of different number of AR and MA terms, from $VARMA(0,1)$ and $VARMA(1,0)$ to $VARMA(6,6)$ and all the combinations in-between.
After ...
- 329
2
votes
1
answer
22
views
Can we have a large negative Loglikehood function and a small positive BIC?
I am currently reading a paper and the autors end up with a log-likehood (llf) of around -4600 and a BIC of around 5,4.
This may come down to my comprehension of the BIC, but I thought that
BIC = -2 * ...
- 41
0
votes
0
answers
31
views
best negative BIC in model selection [duplicate]
I am trying to select the best model with the library broom in R. The script is something like this:
...
0
votes
0
answers
50
views
correct degrees of freedom
I want to compare two models via BIC in a regression. My conditional error distribution is gaussian. The existing model (not changeable) is:
model 1:
...
- 83
1
vote
1
answer
137
views
Compare linear regression slopes between non-nested models with differing dataset sizes
I'd like to test if the slopes of two linear regression models differ. However, the caveat is that one of the regressions fits a subset of the data, and the other fits the whole dataset. The two ...
- 135
0
votes
1
answer
78
views
Negative Infinity AIC and BIC
I was trying to compare best fit model for monthly precipitation data sets and negative and positive infinity (-inf and inf) as values have showed up for both AIC and BIC tests. Can anyone tell me ...
0
votes
1
answer
49
views
Best-Subset Regression based on BIC versus Forward Selection based on AIC
I am trying to get a better grasp of BIC and AIC scores. I know BIC has a harsher penalty than AIC regarding model size (it prefers smaller, less complex models).
Suppose there is a situation where I ...
- 1
0
votes
0
answers
22
views
Is there a universal way to calculate model likelihoods for an arbitrary distribution?
I have no background in statistics but have been tasked to use AIC and BIC to select a model for some observed experimental data. The population data cannot be assumed to obey any particular ...
- 1
0
votes
0
answers
92
views
Computing BIC and other methods to find most likely K cluster
I have recently been diving into the world of population genetics and still remain with some questions when trying to conclude what the most likely value of K for my dataset would be.
To give you some ...
0
votes
0
answers
120
views
AIC/BIC Based Model Selection And Sample Size
I am using BIC to tune a lasso estimation and select the features that will be used in further analysis. The data is quite large, and I have some prior domain knowledge on it, so I split it by several ...
- 71
5
votes
2
answers
554
views
How to interpret negative values for -2LL, AIC, and BIC?
From a statistical point of view, can -2LL, AIC, and BIC in the table of information criteria from SPSS output be less than zero, ie negative? In this case, how should they be interpreted?
Note: These ...
- 71
1
vote
1
answer
355
views
The BIC is different in cph and coxph for the same model
I constructed the same cox regression models by using cph in rms package and coxph in R, but when I compared the two models with BIC, I got 4086.559 for coxph, 4114.43 for cph. But the two models both ...
- 143
2
votes
1
answer
122
views
Is BIC asymptotically efficient for minimizing prediction error if the true model is being considered?
If a set of models is being compared using BIC and AIC, given the fact that the true model (the one which generated the data) is in this set (and given the other assumptions that guarantee BIC ...
- 550
1
vote
1
answer
1k
views
Calculating AIC & BIC
I have an output from two LMER-models and I'd like to calculate AIC & BIC. I believe I've understood the tables correctly, but I'm uncertain regarding the k parameter; have I understood it ...
- 93
1
vote
0
answers
31
views
Log Likelihood for Two Stage Least Squares (for AIC or BIC)
I'm looking to use a number of information criteria (BIC, AIC, etc.) for Two Stage Least Squares.
Of course all information criteria need a log likelihood - and I'm also aware that for a Log-...
- 133
0
votes
1
answer
89
views
Selecting a model for linear regression: adjusted metrics (BIC, AIC, adjusted R2 etc...) on training set or validation/crossvalidation using test set?
Linear regression has model hyperparameters such as number of predictors. For example in a autoregressive time series model AR(p), p is the number of predictors.
To find which value of p to find we ...
- 31
0
votes
0
answers
579
views
negative values for AIC and BIC
I am trying to fit a gumbel distribution using MLE for the following 10 data points.
DATA=(3.62,3.76,3.57,3.56,3.61,3.77,3.46,3.6,3.39,3.74)
The problem is that the ...
- 1
0
votes
0
answers
66
views
Picking a suitable performance metric when comparing the same model but using different sets of training data (Causal inference model)
I am comparing the same models prediction accuracy (Causal Impact) using different control variables as predictors and looking for a metric to decide which set of controls to use. Reading into AIC and ...
- 1
1
vote
0
answers
114
views
BIC and RMSE are Contradict each other for ARIMA Model Selection using R: Do I Err in Theory or in Practice?
I was surprised to see that RMSE and BIC have contradictory trends for the same time-series data.
EDITED
The procedures in my code are:
simulate a 15 AR series of ...
- 117
0
votes
2
answers
176
views
Bayes Information Criterion — what does log mean?
Super basic question about the BIC — is it defined in terms of log base ten or the natural logarithm? I see the latter on Wikipedia; but see ‘log’ not ‘ln’ in the original paper (though am aware that ...
- 1
1
vote
1
answer
3k
views
How to calculate AIC and BIC?
I should find formula of BIC and AIC which is used in statsmodels.
I have array with values:
...
0
votes
0
answers
58
views
How can I determine what value of k to use for an AIC/BIC of a fractional power equation?
I have two equations of which I am trying to determine which is the better fit using AIC and BIC: a quadratic equation of the formula
$$\ y = β_{1}x^2+β_{2}x+β_{0}$$
and a fractional power equation ...
- 181
1
vote
1
answer
169
views
Validity of BIC for Dirichlet process mixture models
I am implementing clustering using Dirichlet process mixture models via scikit learn's Variational Bayesian Gaussian Mixture model. I arrived at the appropriate priors iteratively, and I am able to ...
- 63
1
vote
0
answers
28
views
AIC and BIC for wrong models! [closed]
If the models we consider for the data are wrong, what happens to the model selection with AIC or BIC as the data increases?
Any hint?
- 121
0
votes
1
answer
55
views
How do I find Schwartz criterion (or Bayesian Information Criterion) for these three models?
I have to find the schwarz criterion for each of the models in this maths question using RStudio but I don't know where to start. I know I need to find the free parameters but don't know how to find ...
3
votes
1
answer
2k
views
Can I use AIC/BIC to compare a Poisson model to a negative Binomial model?
I would please like to enquire if it's appropriate for me to compare the fit of a Poisson vs. a negative Binomial model for my data, given that the two models are nested, i.e. the negative Binomial ...
1
vote
1
answer
60
views
Conjectures regarding EM approximations of mixtures of multivariate normal distributions
Consider $X\in\mathbb{R}^{N\times d}$ containing data for $N$ points in $d$ dimensions drawn from a bimodal multivariate normal distribution, where any row $x$ of $X$ follows the mixed multivariate ...
- 2,044
1
vote
1
answer
401
views
How to get log-likelihood from squared deviance in Scikit Learn
The score() function computes D^2, the percentage of deviance explained, but I'd like to get the log-likelihood to calculate BIC. What's the formula to go from deviance to log-likelihood?
Score ...
- 113
1
vote
1
answer
40
views
Test to select best models in production
I've got four models in production and using the average of them as the served prediction. We get ground truth data immediately.
I've optimized them and found the best models during my training/...
1
vote
1
answer
377
views
AIC/BIC vs the rule of "must include lower order interaction"
I am running a series of mixed effect models, which include both linear and quadratic term of a variable T (continuous) and the main IV I (categorical), and facing a dilemma.
Model 2 include ...
- 71