Questions tagged [bic]
BIC is an acronym for Bayesian Information Criterion. BIC is one method of model comparison. See also AIC
305
questions
2
votes
2
answers
344
views
AIC, BIC and log likelihood which more important?
I am currently searching for the best ARMA(p,q) model for my conditional mean.
When comparing the AIC, BIC and LL, I see that some model perform better in AIC, some in BIC and some in LL. The AC and ...
- 41
0
votes
0
answers
18
views
Model selection and validation of latent class models with sampling weights
We are conducting a LCA with sampling weights, using Stata. Since we apply these sampling weights, the dataset becomes very large and the goodness of fit tests do not validate the model and do not ...
- 1
1
vote
0
answers
38
views
Distribution of Bayesian Information Criteria - using BIC when there are multiple datasets
I have two competing nested models : $M1$ and $M2$.
$M1$ has way less parameters.
I know that $M2$ is definitely a true model for data (i.e., can explain data fully) but I claim that it is not the ...
- 31
0
votes
0
answers
12
views
Using AIC/BIC to compare models with and without mediators
Our team ran into an issue where we had some confusion as to whether it's appropriate to use AIC or BIC to compare sets of models with and without mediators. That is, in our first model(s) we only ...
- 1
2
votes
1
answer
111
views
How do you write the AIC and BIC of a regression model in terms of the coefficient-of-determination?
This question is to give a general exposition of the relationship between goodness-of-fit statistics in regression analysis, to answer questions like this one.
Consider a nonlinear Gaussian regression ...
- 94.5k
2
votes
1
answer
107
views
Linear regression gives a good $R^2$ but also a high BIC
I tried a linear model with interactions on my data ($n=95,840$ rows) :
...
- 131
10
votes
4
answers
857
views
In what applications do we prefer Model Selection over Model Averaging?
I'm wondering in what applications or scenarios (or in trying to answer what kind of questions), the researcher would prefer using Model Selection (such as AIC or BIC) over Model Averaging (such as ...
- 1,682
0
votes
0
answers
10
views
What is the number of free parameters in an n-component GMM?
I am trying to calculate BIC = -2logL + log(N)d where d is the number of free parameters or degrees of freedom.
If I am fitting guassian mixture model to the data, ...
- 1
0
votes
1
answer
41
views
Regression curve with lowest combined AIC and BIC is a poor predictor
For fun, I was trying to make a predictor for how long it would take for George R. R. Martin's The Winds of Winter to be released. My "best" model is the one that had the lowest combined AIC ...
- 121
0
votes
0
answers
24
views
Should I use AIC / BIC or rather cross validation for discovering gov. equations through linear regression (SINDy)?
I want to use linear regression with very large design matrix for discovery of governing equations to i.e. physical systems. The design matrix would include potential terms that can be part of the ...
- 11
0
votes
0
answers
18
views
ODE model selection criterion selection criteria
There is substantial literature on model selection criteria and many questions around this topic on CrossValidated. However, I could not find one that covers my case.
I have a time series dataset (of ...
- 11
5
votes
0
answers
82
views
Brief characterizations of AIC and BIC: how helpful are they?
I have found the following one-sentence characterizations of AIC and BIC in a lecture note:
AIC estimates the degree to which the predictive accuracy of the model will generalize to new data.
BIC ...
- 55.5k
1
vote
1
answer
32
views
Interpreting why a VAR produces lower error than VARMA?
I trained various VARMA models on the same dataset consisting of different number of AR and MA terms, from $VARMA(0,1)$ and $VARMA(1,0)$ to $VARMA(6,6)$ and all the combinations in-between.
After ...
- 299
2
votes
1
answer
21
views
Can we have a large negative Loglikehood function and a small positive BIC?
I am currently reading a paper and the autors end up with a log-likehood (llf) of around -4600 and a BIC of around 5,4.
This may come down to my comprehension of the BIC, but I thought that
BIC = -2 * ...
- 41
0
votes
0
answers
25
views
best negative BIC in model selection [duplicate]
I am trying to select the best model with the library broom in R. The script is something like this:
...
0
votes
0
answers
34
views
correct degrees of freedom
I want to compare two models via BIC in a regression. My conditional error distribution is gaussian. The existing model (not changeable) is:
model 1:
...
- 83
1
vote
1
answer
62
views
Compare linear regression slopes between non-nested models with differing dataset sizes
I'd like to test if the slopes of two linear regression models differ. However, the caveat is that one of the regressions fits a subset of the data, and the other fits the whole dataset. The two ...
- 135
0
votes
1
answer
51
views
Negative Infinity AIC and BIC
I was trying to compare best fit model for monthly precipitation data sets and negative and positive infinity (-inf and inf) as values have showed up for both AIC and BIC tests. Can anyone tell me ...
0
votes
1
answer
40
views
Best-Subset Regression based on BIC versus Forward Selection based on AIC
I am trying to get a better grasp of BIC and AIC scores. I know BIC has a harsher penalty than AIC regarding model size (it prefers smaller, less complex models).
Suppose there is a situation where I ...
- 1
0
votes
0
answers
21
views
Is there a universal way to calculate model likelihoods for an arbitrary distribution?
I have no background in statistics but have been tasked to use AIC and BIC to select a model for some observed experimental data. The population data cannot be assumed to obey any particular ...
- 1
0
votes
0
answers
63
views
Computing BIC and other methods to find most likely K cluster
I have recently been diving into the world of population genetics and still remain with some questions when trying to conclude what the most likely value of K for my dataset would be.
To give you some ...
0
votes
0
answers
90
views
AIC/BIC Based Model Selection And Sample Size
I am using BIC to tune a lasso estimation and select the features that will be used in further analysis. The data is quite large, and I have some prior domain knowledge on it, so I split it by several ...
- 71
5
votes
2
answers
449
views
How to interpret negative values for -2LL, AIC, and BIC?
From a statistical point of view, can -2LL, AIC, and BIC in the table of information criteria from SPSS output be less than zero, ie negative? In this case, how should they be interpreted?
Note: These ...
- 71
0
votes
0
answers
9
views
Does variable selection consistency in lasso implicate anything about the BIC-selected results?
In the theory of lasso and its variations, researchers have proved the variable selection consistency (e.g., Theorem 1 in the paper from JMLR2006 on the variable selection of lasso) under various ...
- 15
1
vote
1
answer
232
views
The BIC is different in cph and coxph for the same model
I constructed the same cox regression models by using cph in rms package and coxph in R, but when I compared the two models with BIC, I got 4086.559 for coxph, 4114.43 for cph. But the two models both ...
- 101
2
votes
1
answer
99
views
Is BIC asymptotically efficient for minimizing prediction error if the true model is being considered?
If a set of models is being compared using BIC and AIC, given the fact that the true model (the one which generated the data) is in this set (and given the other assumptions that guarantee BIC ...
- 550
0
votes
0
answers
14
views
Problems with training a discriminative model with generative methods
I'm reading Friedman, Geiger and Goldszmidt's 1997 paper "Bayesian Network Classifiers".
On page 7, they discuss the problems with using methods for learning generative models (specifically ...
- 23
1
vote
1
answer
627
views
Calculating AIC & BIC
I have an output from two LMER-models and I'd like to calculate AIC & BIC. I believe I've understood the tables correctly, but I'm uncertain regarding the k parameter; have I understood it ...
- 81
1
vote
0
answers
31
views
Log Likelihood for Two Stage Least Squares (for AIC or BIC)
I'm looking to use a number of information criteria (BIC, AIC, etc.) for Two Stage Least Squares.
Of course all information criteria need a log likelihood - and I'm also aware that for a Log-...
- 133
0
votes
1
answer
75
views
Selecting a model for linear regression: adjusted metrics (BIC, AIC, adjusted R2 etc...) on training set or validation/crossvalidation using test set?
Linear regression has model hyperparameters such as number of predictors. For example in a autoregressive time series model AR(p), p is the number of predictors.
To find which value of p to find we ...
- 31
0
votes
0
answers
400
views
negative values for AIC and BIC
I am trying to fit a gumbel distribution using MLE for the following 10 data points.
DATA=(3.62,3.76,3.57,3.56,3.61,3.77,3.46,3.6,3.39,3.74)
The problem is that the ...
- 1
0
votes
0
answers
60
views
Picking a suitable performance metric when comparing the same model but using different sets of training data (Causal inference model)
I am comparing the same models prediction accuracy (Causal Impact) using different control variables as predictors and looking for a metric to decide which set of controls to use. Reading into AIC and ...
- 1
1
vote
0
answers
78
views
BIC and RMSE are Contradict each other for ARIMA Model Selection using R: Do I Err in Theory or in Practice?
I was surprised to see that RMSE and BIC have contradictory trends for the same time-series data.
EDITED
The procedures in my code are:
simulate a 15 AR series of ...
- 107
0
votes
2
answers
145
views
Bayes Information Criterion — what does log mean?
Super basic question about the BIC — is it defined in terms of log base ten or the natural logarithm? I see the latter on Wikipedia; but see ‘log’ not ‘ln’ in the original paper (though am aware that ...
- 1
1
vote
1
answer
2k
views
How to calculate AIC and BIC?
I should find formula of BIC and AIC which is used in statsmodels.
I have array with values:
...
0
votes
0
answers
48
views
How can I determine what value of k to use for an AIC/BIC of a fractional power equation?
I have two equations of which I am trying to determine which is the better fit using AIC and BIC: a quadratic equation of the formula
$$\ y = β_{1}x^2+β_{2}x+β_{0}$$
and a fractional power equation ...
- 181
1
vote
1
answer
140
views
Validity of BIC for Dirichlet process mixture models
I am implementing clustering using Dirichlet process mixture models via scikit learn's Variational Bayesian Gaussian Mixture model. I arrived at the appropriate priors iteratively, and I am able to ...
- 63
1
vote
0
answers
27
views
AIC and BIC for wrong models! [closed]
If the models we consider for the data are wrong, what happens to the model selection with AIC or BIC as the data increases?
Any hint?
- 121
0
votes
1
answer
52
views
How do I find Schwartz criterion (or Bayesian Information Criterion) for these three models?
I have to find the schwarz criterion for each of the models in this maths question using RStudio but I don't know where to start. I know I need to find the free parameters but don't know how to find ...
3
votes
1
answer
1k
views
Can I use AIC/BIC to compare a Poisson model to a negative Binomial model?
I would please like to enquire if it's appropriate for me to compare the fit of a Poisson vs. a negative Binomial model for my data, given that the two models are nested, i.e. the negative Binomial ...
0
votes
1
answer
53
views
Conjectures regarding EM approximations of mixtures of multivariate normal distributions
Consider $X\in\mathbb{R}^{N\times d}$ containing data for $N$ points in $d$ dimensions drawn from a bimodal multivariate normal distribution, where any row $x$ of $X$ follows the mixed multivariate ...
- 1,663
1
vote
1
answer
348
views
How to get log-likelihood from squared deviance in Scikit Learn
The score() function computes D^2, the percentage of deviance explained, but I'd like to get the log-likelihood to calculate BIC. What's the formula to go from deviance to log-likelihood?
Score ...
- 113
1
vote
1
answer
36
views
Test to select best models in production
I've got four models in production and using the average of them as the served prediction. We get ground truth data immediately.
I've optimized them and found the best models during my training/...
1
vote
1
answer
287
views
AIC/BIC vs the rule of "must include lower order interaction"
I am running a series of mixed effect models, which include both linear and quadratic term of a variable T (continuous) and the main IV I (categorical), and facing a dilemma.
Model 2 include ...
- 81
8
votes
1
answer
121
views
AIC/BIC formula wrong in James/Witten?
Reading "An Introduction to
Statistical Learning" (by James, Witten, Hastie and Tibshirani), on p.211 I came across the following formula for BIC in case of linear regression:
$ BIC = \frac{...
- 531
0
votes
1
answer
551
views
ARIMA grid search for best $p$, $d$, $q$ orders based on BIC metric
i am new to R, and i was hoping to get some answers to some questions i have
is there a function to loop through a range of ARIMA model to get the best order/seasonal based on BIC/AIC ?
do i have ...
- 11
1
vote
1
answer
438
views
Calculate AIC value from a given BIC value?
Is it valid, given a certain BIC value (an output from an R package) with known n and k, to transform the BIC value via mathematical manipulation of the formulae for BIC and AIC to get the "...
- 4,599
5
votes
1
answer
114
views
The Ambiguity in Schwarz Information Criterion Definition
Suppose there are 100 countries, $i = 1, 2, ..., 100$.
Let $b_i$ be the median birth weight of all new born boys in country #i in 2019.
Let $g_i$ be the median birth weight of all new born girls in ...
- 431
1
vote
1
answer
156
views
Differences between formulas for AIC and BIC
I have a question regarding the information criteria AIC and BIC:
I found different formulas for the AIC/BIC, the common ones including the likelihood $\mathcal{L}$ are
$$AIC = 2K - 2 ln(\mathcal{L})\...
- 53
1
vote
1
answer
744
views
Log-Likelihood Computation for AIC & BIC
Considering $n$ observations that an be modelled by a Gaussian error model and two nested motion models with $p = 4$ and $p = 7$ parameters, I want to compute the log likelihoods $L$ given the Maximum ...
- 33