Questions tagged [bic]

BIC is an acronym for Bayesian Information Criterion. BIC is one method of model comparison. See also AIC

Filter by
Sorted by
Tagged with
1 vote
1 answer
90 views

How to use priors on the parameter number with an information criterion (AIC, BIC, …)?

Example The example is made up because I hope that it’s more accessible than my actual problem. I want to determine the number of planets of a star. I have: data for some astronomical observable of ...
  • 2,257
2 votes
0 answers
42 views

Is it circular reasoning to compute the ELBO using MCMC?

Let's say we have a posterior distribution $q(\theta) = p(\theta \mid D, \mathcal{M})$ over parameters $\theta$ given data $D$ and a model $\mathcal{M}$. As is often the case, computing $q$ is hard, ...
  • 21
0 votes
0 answers
19 views

Difference between Bayesian Information Criteria and Approximate Bayesian Computation as model selection

My question is not very technical and more like a discussion but I will be happy to have a technical input for the comparison b/w BIC and ABC. I am trying to understand and use the best model ...
3 votes
1 answer
53 views

Model selection criteria that represent a compromise between AIC and BIC

I am very familiar with the ideas and formula of the two popular model selection criteria AIC/AICc and BIC. When I use them for practical problems in chemometrics, the use of AIC/AICc often gives ...
  • 33
0 votes
0 answers
66 views

SEM model comparison

I have problem choosing a better measurement model between two (i.e., A and B). Both models have acceptable fit indices. For example, the RMSEA of A is .056; the RMSEA of B is .067. By RMSEA, A seems ...
0 votes
1 answer
68 views

AIC and BIC in GEE

Can I apply Akaike’s information criterion (AIC) and Quasi-likelihood under the independence model information criterion (QIC) in GEE?
2 votes
0 answers
33 views

AIC vs BIC for time series clustering and descriptive purposes

I'm in the process of fitting a hidden markov model with gaussian mixtures to time series health data. The primary purpose of this is descriptive, not predictive – I'm using the fitted model to give a ...
  • 21
2 votes
2 answers
598 views

AIC, BIC and log likelihood which more important?

I am currently searching for the best ARMA(p,q) model for my conditional mean. When comparing the AIC, BIC and LL, I see that some model perform better in AIC, some in BIC and some in LL. The AC and ...
  • 41
0 votes
0 answers
48 views

Model selection and validation of latent class models with sampling weights

We are conducting a LCA with sampling weights, using Stata. Since we apply these sampling weights, the dataset becomes very large and the goodness of fit tests do not validate the model and do not ...
  • 1
1 vote
0 answers
48 views

Distribution of Bayesian Information Criteria - using BIC when there are multiple datasets

I have two competing nested models : $M1$ and $M2$. $M1$ has way less parameters. I know that $M2$ is definitely a true model for data (i.e., can explain data fully) but I claim that it is not the ...
  • 31
0 votes
0 answers
32 views

Using AIC/BIC to compare models with and without mediators

Our team ran into an issue where we had some confusion as to whether it's appropriate to use AIC or BIC to compare sets of models with and without mediators. That is, in our first model(s) we only ...
  • 1
2 votes
1 answer
206 views

How do you write the AIC and BIC of a regression model in terms of the coefficient-of-determination?

This question is to give a general exposition of the relationship between goodness-of-fit statistics in regression analysis, to answer questions like this one. Consider a nonlinear Gaussian regression ...
  • 102k
2 votes
1 answer
136 views

Linear regression gives a good $R^2$ but also a high BIC

I tried a linear model with interactions on my data ($n=95,840$ rows): ...
  • 131
10 votes
4 answers
1k views

In what applications do we prefer Model Selection over Model Averaging?

I'm wondering in what applications or scenarios (or in trying to answer what kind of questions), the researcher would prefer using Model Selection (such as AIC or BIC) over Model Averaging (such as ...
  • 1,898
0 votes
0 answers
58 views

What is the number of free parameters in an n-component GMM?

I am trying to calculate BIC = -2logL + log(N)d where d is the number of free parameters or degrees of freedom. If I am fitting guassian mixture model to the data, ...
  • 1
0 votes
1 answer
45 views

Regression curve with lowest combined AIC and BIC is a poor predictor

For fun, I was trying to make a predictor for how long it would take for George R. R. Martin's The Winds of Winter to be released. My "best" model is the one that had the lowest combined AIC ...
  • 121
0 votes
0 answers
32 views

Should I use AIC / BIC or rather cross validation for discovering gov. equations through linear regression (SINDy)?

I want to use linear regression with very large design matrix for discovery of governing equations to i.e. physical systems. The design matrix would include potential terms that can be part of the ...
  • 11
5 votes
2 answers
219 views

How to calculate the degrees of freedom for L1 and L2 regularised GLMs?

My goal is to calculate various information criteria for generalised linear models (e.g., the AIC). To do this, we need to calculate the effective degrees of freedom of the trained model. In an ...
  • 151
0 votes
0 answers
31 views

ODE model selection criterion selection criteria

There is substantial literature on model selection criteria and many questions around this topic on CrossValidated. However, I could not find one that covers my case. I have a time series dataset (of ...
  • 11
5 votes
0 answers
84 views

Brief characterizations of AIC and BIC: how helpful are they?

I have found the following one-sentence characterizations of AIC and BIC in a lecture note: AIC estimates the degree to which the predictive accuracy of the model will generalize to new data. BIC ...
1 vote
1 answer
46 views

Interpreting why a VAR produces lower error than VARMA?

I trained various VARMA models on the same dataset consisting of different number of AR and MA terms, from $VARMA(0,1)$ and $VARMA(1,0)$ to $VARMA(6,6)$ and all the combinations in-between. After ...
  • 329
2 votes
1 answer
22 views

Can we have a large negative Loglikehood function and a small positive BIC?

I am currently reading a paper and the autors end up with a log-likehood (llf) of around -4600 and a BIC of around 5,4. This may come down to my comprehension of the BIC, but I thought that BIC = -2 * ...
0 votes
0 answers
31 views

best negative BIC in model selection [duplicate]

I am trying to select the best model with the library broom in R. The script is something like this: ...
0 votes
0 answers
50 views

correct degrees of freedom

I want to compare two models via BIC in a regression. My conditional error distribution is gaussian. The existing model (not changeable) is: model 1: ...
  • 83
1 vote
1 answer
137 views

Compare linear regression slopes between non-nested models with differing dataset sizes

I'd like to test if the slopes of two linear regression models differ. However, the caveat is that one of the regressions fits a subset of the data, and the other fits the whole dataset. The two ...
  • 135
0 votes
1 answer
78 views

Negative Infinity AIC and BIC

I was trying to compare best fit model for monthly precipitation data sets and negative and positive infinity (-inf and inf) as values have showed up for both AIC and BIC tests. Can anyone tell me ...
0 votes
1 answer
49 views

Best-Subset Regression based on BIC versus Forward Selection based on AIC

I am trying to get a better grasp of BIC and AIC scores. I know BIC has a harsher penalty than AIC regarding model size (it prefers smaller, less complex models). Suppose there is a situation where I ...
  • 1
0 votes
0 answers
22 views

Is there a universal way to calculate model likelihoods for an arbitrary distribution?

I have no background in statistics but have been tasked to use AIC and BIC to select a model for some observed experimental data. The population data cannot be assumed to obey any particular ...
0 votes
0 answers
92 views

Computing BIC and other methods to find most likely K cluster

I have recently been diving into the world of population genetics and still remain with some questions when trying to conclude what the most likely value of K for my dataset would be. To give you some ...
0 votes
0 answers
120 views

AIC/BIC Based Model Selection And Sample Size

I am using BIC to tune a lasso estimation and select the features that will be used in further analysis. The data is quite large, and I have some prior domain knowledge on it, so I split it by several ...
5 votes
2 answers
554 views

How to interpret negative values for -2LL, AIC, and BIC?

From a statistical point of view, can -2LL, AIC, and BIC in the table of information criteria from SPSS output be less than zero, ie negative? In this case, how should they be interpreted? Note: These ...
1 vote
1 answer
355 views

The BIC is different in cph and coxph for the same model

I constructed the same cox regression models by using cph in rms package and coxph in R, but when I compared the two models with BIC, I got 4086.559 for coxph, 4114.43 for cph. But the two models both ...
  • 143
2 votes
1 answer
122 views

Is BIC asymptotically efficient for minimizing prediction error if the true model is being considered?

If a set of models is being compared using BIC and AIC, given the fact that the true model (the one which generated the data) is in this set (and given the other assumptions that guarantee BIC ...
1 vote
1 answer
1k views

Calculating AIC & BIC

I have an output from two LMER-models and I'd like to calculate AIC & BIC. I believe I've understood the tables correctly, but I'm uncertain regarding the k parameter; have I understood it ...
  • 93
1 vote
0 answers
31 views

Log Likelihood for Two Stage Least Squares (for AIC or BIC)

I'm looking to use a number of information criteria (BIC, AIC, etc.) for Two Stage Least Squares. Of course all information criteria need a log likelihood - and I'm also aware that for a Log-...
0 votes
1 answer
89 views

Selecting a model for linear regression: adjusted metrics (BIC, AIC, adjusted R2 etc...) on training set or validation/crossvalidation using test set?

Linear regression has model hyperparameters such as number of predictors. For example in a autoregressive time series model AR(p), p is the number of predictors. To find which value of p to find we ...
0 votes
0 answers
579 views

negative values for AIC and BIC

I am trying to fit a gumbel distribution using MLE for the following 10 data points. DATA=(3.62,3.76,3.57,3.56,3.61,3.77,3.46,3.6,3.39,3.74) The problem is that the ...
  • 1
0 votes
0 answers
66 views

Picking a suitable performance metric when comparing the same model but using different sets of training data (Causal inference model)

I am comparing the same models prediction accuracy (Causal Impact) using different control variables as predictors and looking for a metric to decide which set of controls to use. Reading into AIC and ...
1 vote
0 answers
114 views

BIC and RMSE are Contradict each other for ARIMA Model Selection using R: Do I Err in Theory or in Practice?

I was surprised to see that RMSE and BIC have contradictory trends for the same time-series data. EDITED The procedures in my code are: simulate a 15 AR series of ...
0 votes
2 answers
176 views

Bayes Information Criterion — what does log mean?

Super basic question about the BIC — is it defined in terms of log base ten or the natural logarithm? I see the latter on Wikipedia; but see ‘log’ not ‘ln’ in the original paper (though am aware that ...
  • 1
1 vote
1 answer
3k views

How to calculate AIC and BIC?

I should find formula of BIC and AIC which is used in statsmodels. I have array with values: ...
0 votes
0 answers
58 views

How can I determine what value of k to use for an AIC/BIC of a fractional power equation?

I have two equations of which I am trying to determine which is the better fit using AIC and BIC: a quadratic equation of the formula $$\ y = β_{1}x^2+β_{2}x+β_{0}$$ and a fractional power equation ...
1 vote
1 answer
169 views

Validity of BIC for Dirichlet process mixture models

I am implementing clustering using Dirichlet process mixture models via scikit learn's Variational Bayesian Gaussian Mixture model. I arrived at the appropriate priors iteratively, and I am able to ...
  • 63
1 vote
0 answers
28 views

AIC and BIC for wrong models! [closed]

If the models we consider for the data are wrong, what happens to the model selection with AIC or BIC as the data increases? Any hint?
  • 121
0 votes
1 answer
55 views

How do I find Schwartz criterion (or Bayesian Information Criterion) for these three models?

I have to find the schwarz criterion for each of the models in this maths question using RStudio but I don't know where to start. I know I need to find the free parameters but don't know how to find ...
3 votes
1 answer
2k views

Can I use AIC/BIC to compare a Poisson model to a negative Binomial model?

I would please like to enquire if it's appropriate for me to compare the fit of a Poisson vs. a negative Binomial model for my data, given that the two models are nested, i.e. the negative Binomial ...
1 vote
1 answer
60 views

Conjectures regarding EM approximations of mixtures of multivariate normal distributions

Consider $X\in\mathbb{R}^{N\times d}$ containing data for $N$ points in $d$ dimensions drawn from a bimodal multivariate normal distribution, where any row $x$ of $X$ follows the mixed multivariate ...
1 vote
1 answer
401 views

How to get log-likelihood from squared deviance in Scikit Learn

The score() function computes D^2, the percentage of deviance explained, but I'd like to get the log-likelihood to calculate BIC. What's the formula to go from deviance to log-likelihood? Score ...
  • 113
1 vote
1 answer
40 views

Test to select best models in production

I've got four models in production and using the average of them as the served prediction. We get ground truth data immediately. I've optimized them and found the best models during my training/...
1 vote
1 answer
377 views

AIC/BIC vs the rule of "must include lower order interaction"

I am running a series of mixed effect models, which include both linear and quadratic term of a variable T (continuous) and the main IV I (categorical), and facing a dilemma. Model 2 include ...

1
2 3 4 5
7