Questions tagged [bic]

BIC is an acronym for Bayesian Information Criterion. BIC is one method of model comparison. See also AIC

Filter by
Sorted by
Tagged with
273 votes
13 answers
214k views

Is there any reason to prefer the AIC or BIC over the other?

The AIC and BIC are both methods of assessing model fit penalized for the number of estimated parameters. As I understand it, BIC penalizes models more for free parameters than does AIC. Beyond a ...
user avatar
26 votes
3 answers
6k views

Paradox in model selection (AIC, BIC, to explain or to predict?)

Having read Galit Shmueli's "To Explain or to Predict" (2010) and some literature on model selection using AIC and BIC, I am puzzled by an apparent contradiction. There are three premises, AIC- ...
user avatar
12 votes
2 answers
21k views

Compute BIC clustering criterion (to validate clusters after K-means)

I'm wondering if there is a good way to calculate the clustering criterion based on BIC formula, for a k-means output in R? I'm a bit confused as to how to calculate that BIC so that I can compare it ...
user avatar
47 votes
5 answers
210k views

AIC guidelines in model selection

I typically use BIC as my understanding is that it values parsimony more strongly than does AIC. However, I have decided to use a more comprehensive approach now and would like to use AIC as well. I ...
user avatar
4 votes
2 answers
4k views

AIC/BIC and data transformation

Can you use AIC/BIC to compare models on untransformed data with models on transformed data (such as log, inverse hyperbolic sine, etc.)? I.e. if a model using logged data gives an AIC = 53.62 and a ...
user avatar
6 votes
2 answers
2k views

What is the actual significance of a difference in AIC or BIC values?

Usually, when a difference of a statistic is discussed, that discussion is presented in the context of a significance of that difference. When self-entropy, i.e., information content, is examined, ...
user avatar
  • 11.9k
50 votes
3 answers
5k views

AIC,BIC,CIC,DIC,EIC,FIC,GIC,HIC,IIC --- Can I use them interchangeably?

On p. 34 of his PRNN Brian Ripley comments that "The AIC was named by Akaike (1974) as 'An Information Criterion' although it seems commonly believed that the A stands for Akaike". Indeed, when ...
user avatar
  • 3,773
16 votes
3 answers
30k views

Using BIC to estimate the number of k in KMEANS

I am currently trying to compute the BIC for my toy data set (ofc iris (: ). I want to reproduce the results as shown here (Fig. 5). That paper is also my source for the BIC formulas. I have 2 ...
user avatar
  • 520
36 votes
3 answers
20k views

Is it possible to calculate AIC and BIC for lasso regression models?

Is it possible to calculate AIC or BIC values for lasso regression models and other regularized models where parameters are only partially entering the equation. How does one determine the degrees of ...
user avatar
  • 864
15 votes
1 answer
5k views

Variable selection vs Model selection

So I understand that variable selection is a part of model selection. But what exactly does model selection consist of? Is it more than the following: 1) choose a distribution for your model 2) ...
user avatar
  • 1,444
7 votes
2 answers
2k views

If the AIC and the BIC are asymptotically equivalent to cross validation, is it possible to dispense with a test set when using them?

Several sources I've come across state that the AIC and the BIC are asymptotically equivalent to cross-validation (see multiple answers here for example, and here), . When training a predictive ...
user avatar
  • 10.9k
13 votes
1 answer
10k views

Mclust model selection

The R package mclust uses BIC as a criteria for cluster model selection. From my understanding, a model with the lowest BIC should be selected over other models (if ...
user avatar
  • 2,230
6 votes
1 answer
3k views

X-mean algorithm BIC calculation question

I'm having trouble understanding some of the formulas in this paper related to BIC calculation (Dan Pelleg and Andrew Moore, X-means: Extending K-means with Efficient Estimation of the Number of ...
user avatar
  • 165
20 votes
1 answer
1k views

Does BIC try to find a true model?

This question is a follow-up or attempt to clear up possible confusion regarding a topic I and many others find a bit difficult, regarding the difference between AIC and BIC. In a very nice answer by @...
user avatar
  • 1,444
11 votes
2 answers
5k views

What is a "Unit Information Prior"?

I've been reading Wagenmakers (2007) A practical solution to the pervasive problem of p values. I'm intrigued by the conversion of BIC values into Bayes factors and probabilities. However, so far I ...
user avatar
10 votes
2 answers
2k views

Why information criterion (not adjusted $R^2$) are used to select appropriate lag order in time series model?

In time series models, like ARMA-GARCH, to select appropriate lag or order of the model different information criterion, like AIC, BIC, SIC etc, are used. My question is very simple, why donot we ...
user avatar
  • 2,180
6 votes
1 answer
2k views

AIC and BIC criterion for Model selection, how is it used in this paper?

I'm reading Model selection and inference: Facts and fiction by Leeb & Pötscher (2005) (link), in this paper they look at an example in linear regression: Let $$Y_i = \alpha x_{i1}+\beta x_{i2}+\...
user avatar
  • 1,142
6 votes
3 answers
9k views

Zero-inflated Poisson regression Vuong test: Raw, AIC- or BIC-corrected results

I'm analyzing count data for a set of ten species and found that for the five species with highest detection rate, the zero-inflated poisson (ZIP) regression fits the data significantly better than ...
user avatar
  • 125
24 votes
1 answer
6k views

Why isn't Akaike information criterion used more in machine learning?

I just ran into "Akaike information criterion", and I noticed this large amount of literature on model selection (also things like BIC seem to exist). Why don't contemporary machine learning methods ...
user avatar
  • 921
13 votes
2 answers
7k views

Number of parameters in Markov model

I want to use BIC for HMM model selection: BIC = -2*logLike + num_of_params * log(num_of_data) So how do I count the number of parameters in the HMM model. ...
user avatar
  • 606
8 votes
1 answer
5k views

When is the AIC a good model selection criterion for forecasting and when is it not?

I'm trying to wrap my head around why the AIC and other similar ICs work as proxies for out of sample error when trying to perform automated forecast generation. So I performed an experiment on the ...
user avatar
  • 10.9k
2 votes
1 answer
3k views

Interpreting AIC and BIC fit

I'm writing a CFA paper, and I have run into some trouble interpreting the AIC and BIC. This is my first paper using continuous variables, thus the first time I will be reporting these fit statistics ...
user avatar
  • 31
4 votes
1 answer
434 views

Post Model Selection Inference problems - which remedies exist?

Recently, Hannes Leeb from Yale University and Benedikt Pötscher from the University of Vienna have published a series of papers dealing with what they call Post Model Selection Inference problems.* ...
user avatar
  • 1,491
5 votes
3 answers
2k views

Comparison between MDL and BIC

I'm currently studying Hidden Markov Models. There's a set of observations from which I need to determine the optimal number of states. After having found the maximum likelihood using Baum-Welch, I ...
user avatar
  • 167
8 votes
1 answer
997 views

Can BIC be Used for Hypothesis Testing

Define the Bayesian information criterion as $$ \mathrm{BIC} = {-2 \cdot \ln{\hat L} + k \cdot (\ln(n) - \ln(2 \pi))} $$ (I do not drop the constant, $ - \ln(2 \pi)$, to avoid issues when equating to ...
user avatar
3 votes
1 answer
3k views

Naive Bayes likelihood

I'm interested in computing the Bayesian Information Criterion (BIC) for a set of Naive Bayes models. The NB can be described as follows, for a two-class $Y \in {0,1}$ with predictors $X = (x_1, x_2, ...
user avatar
  • 2,230
6 votes
3 answers
4k views

In Bayesian Information Criterion (BIC), why does having bigger n get penalized?

The Bayesian Information Criterion (BIC) is calculated with: $$ \text{BIC} = \frac{1}{n \hat{\delta}^2} \Big(\text{RSS} + \ln(n) d \hat{\delta}^2 \Big) $$ where RSS is residual sum of squares and ...
user avatar
6 votes
1 answer
613 views

The unit information prior and its BIC approximation

Just moments ago, I asked this question because I've been reading Wagenmakers 2007. I now have a better understanding of what a unit information prior is and can push my knowledge further with two (by ...
user avatar
1 vote
1 answer
365 views

Using AIC/BIC within cross-validation for likelihood based loss functions

For a course I am teaching, I am having my students fit a Gaussian mixture model using MLEs via the EM algorithm to a bivariate dataset. I have asked the students to use use cross-validation to choose ...
user avatar
  • 14.5k
23 votes
3 answers
37k views

AIC & BIC number interpretation

I am looking for examples of how to interpret AIC (Akaike information criterion) and BIC (Bayesian information criterion) estimates. Can negative difference between BICs be interpreted as the ...
user avatar
  • 231
9 votes
1 answer
1k views

How can the AIC or BIC be used instead of the train/test split?

I've recently come across several "informal" sources that indicate that in some circumstances, if we use the AIC or BIC to train a time series model, we don't need to split the data into test and ...
user avatar
  • 10.9k
6 votes
2 answers
640 views

Sparse parameters when computing AIC, BIC, etc

I'm designing large-scale, regularized logistic regression models with lots of sparse, binarized features. e.g. isUS, isFR, etc. As a result, a lot of the weights in the model are zero. I'm wondering ...
user avatar
  • 933
10 votes
2 answers
2k views

Does there exist a model fit statistic (like AIC or BIC) that can be used for absolute instead of just relative comparisons?

I'm not that familiar with this literature, so please forgive me if this is an obvious question. Since AIC and BIC depend on maximizing the likelihood, it seems that they can only be used to make ...
user avatar
3 votes
1 answer
405 views

Is it inherently invalid to use BIC for model averaging?

If I understand AIC/AICc vs. BIC correctly, AIC presumes that there is no "true" model and that any given model is simply a "best worst approximation". However, BIC presumes that there IS a "true" ...
user avatar
  • 831
1 vote
1 answer
485 views

Where is the divide between information criterion (AIC, BIC, etc...) and cross validation?

I've taken a regression class and am now in a machine learning class. In regression, we talk about model selection using adj-R2 and AIC/BIC. In my machine learning class, we primarily select models ...
user avatar
  • 2,523
1 vote
1 answer
564 views

Bayesian Information Criterion and Basic Marginal likelihood identity

Given the basic marginal likelihood identity: $$ ln \; m(y)=ln \; p(y|\theta^*)+ln\; \pi(\theta^*)-ln \pi(\theta^*|y) $$ is there a way to derive from this the Bayesian Information Criterion?
user avatar
10 votes
1 answer
164 views

AIC/BIC formula wrong in James/Witten?

Reading "An Introduction to Statistical Learning" (by James, Witten, Hastie and Tibshirani), on p.211 I came across the following formula for BIC in case of linear regression: $ BIC = \frac{...
user avatar
  • 551
7 votes
2 answers
11k views

Negative BIC in k-means

Probably a simple question but I'm trying to interpret BIC for k-means. I have some k-means clustering and calculating BIC gives me a negative value, with a plot something like this: ...
user avatar
  • 215
6 votes
1 answer
2k views

Number of components for Gaussian mixture model?

I have a vector of numeric values. My hypothesis is that this vector is a mixture drawn from two Gaussian distributions (ie k = 2). However, it is possible that there is only one Gaussian underlying ...
user avatar
  • 306
2 votes
1 answer
176 views

How do you write the AIC and BIC of a regression model in terms of the coefficient-of-determination?

This question is to give a general exposition of the relationship between goodness-of-fit statistics in regression analysis, to answer questions like this one. Consider a nonlinear Gaussian regression ...
user avatar
  • 100k
1 vote
0 answers
2k views

Correct calculation of BIC (Bayesian Information Criterion) to determine K for K-Means

I am trying to calculate BIC in python. In python, there is no inbuilt library for computing BIC. I referenced the following link to compute variance and BIC further:- Using BIC to estimate the number ...
user avatar
  • 111
1 vote
2 answers
2k views

Which model should be chosen using AICc/BIC and P-value?

I am using GLM for modeling the binary data. For the predictive variables I am using different number of Zernike Basis functions. To test how many Zernike basis functions I need, I am using AICc/BIC ...
user avatar
  • 169
1 vote
1 answer
2k views

Find (or calculate) log-likelihood value, AIC, and BIC for SUR model (for each equation) with systemfit

I have estimated SUR model with systemfit (R package). With the estimated results, I am trying to get logLik, AIC and BIC for ...
user avatar
0 votes
1 answer
241 views

Clarification on Akaike's IC (AIC) and BIC for Expectation Maximization with time-changing parameters

I apologize in advance for the trivial question, but I need a clarification on the following issue. Suppose I have a generic model in state-space form described as $$x_{t+1}=\phi_{t} x_{t}+w_{t+1}$$ $...
user avatar
  • 1,368
5 votes
2 answers
3k views

LCA number of parameters & degrees of freedom

I have a series of physicians' claims submissions. I would like to perform cluster analysis as an exploratory tool to find patterns in how physicians bill based on things like Revenue Codes, Procedure ...
user avatar
  • 351
3 votes
1 answer
2k views

How do I calculate BIC score for HMM?

I am using a bioinformatics software (ChromHMM) on a given multivariate data set. As of now I have 20 models ranging from 20 states to 40 states in each model. Some of the papers I've read regarding ...
user avatar
  • 471
3 votes
1 answer
229 views

Effective sample size: does it depend on the model?

When applying the Bayesian information criterion, one has to use an "effective sample size" in the penalty term. E.g. if observing longitudinal data (e.g. changes in the blood pressure of an ...
user avatar
  • 634
2 votes
1 answer
1k views

What is the BIC prior for Bayesian linear regression?

Is there a prior probability distribution associated with BIC (Bayesian Information Criterion)? I ask this because in the R package BAS there is a linear modeling function that can be called as ...
user avatar
  • 123
2 votes
1 answer
126 views

Linear regression gives a good $R^2$ but also a high BIC

I tried a linear model with interactions on my data ($n=95,840$ rows): ...
user avatar
  • 131
2 votes
2 answers
4k views

Best BIC value for K-means clusters

I am using code from Using BIC to estimate the number of k in KMEANS (answer by Prabhath Nanisetty) to find BIC values for K-means using different number of components. However, using iris dataset, I ...
user avatar
  • 9,193