Questions tagged [bic]

BIC is an acronym for Bayesian Information Criterion. BIC is one method of model comparison. See also AIC

101 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5 votes
0 answers
83 views

Brief characterizations of AIC and BIC: how helpful are they?

I have found the following one-sentence characterizations of AIC and BIC in a lecture note: AIC estimates the degree to which the predictive accuracy of the model will generalize to new data. BIC ...
user avatar
5 votes
1 answer
402 views

Residual Plots for Model Comparison

Other than using AIC and BIC, I'd like to compare two different models via graphical interpretation, therefore my question is, is it possible to compare two regression models with each one having the ...
user avatar
  • 771
5 votes
1 answer
211 views

Match model selection strategies with modelling objectives

I am confused trying to match different model selection strategies with different modelling objectives. (Unfortunately, my confusion is reflected in the length of the post. Please be patient.) Model ...
user avatar
5 votes
0 answers
168 views

Derivation of the BIC

i am trying to self-study / understand the derivation of the BIC. I have studied that: however - it is not quite clear to me how this leads to the formula below. I don't fully understand where the ...
user avatar
  • 2,102
4 votes
0 answers
56 views

A simulation study of linear mixed model

I am reading this paper, a note on BIC in mixed effects models, and I was trying to repeat their simulation study. And I will paste part of the experiment settings here to clarify my question. Now, I ...
user avatar
  • 597
4 votes
0 answers
1k views

AIC/BIC for quantile regression

I am working on Quantile Regression (QR) and want to assess models using goodness of fit (GOF) measures. I have come across the post here, here that says, AIC/BIC can be calculated for QR model ...
user avatar
  • 51
4 votes
0 answers
1k views

AIC or BIC for robust regression?

I want to fit a robust regression method to my data because there are some outliers that might influence the estimates too much. Now my question: Are criterions like AIC or BIC still useful for robust ...
user avatar
  • 251
4 votes
1 answer
180 views

Sample size when fitting categorical survey data

I have a model which fits data from repeated surveys: at time $t$, a number $n_t$ respondents is asked a question and can give one of $K$ answers ($k=1, ..., K$). This is repeated $T$ times ($t = 1, .....
user avatar
  • 634
4 votes
0 answers
590 views

GLMM with 2 insignificant variables has lower AIC or BIC compared to same model without those variables...?

Background This post has been heavily edited from its previous version (three months ago). I am investigating habitat selection of 35 territorial wolves over several years of denning seasons (41 ...
user avatar
  • 525
3 votes
0 answers
81 views

Does a high Chi square p-value for a whole model mean it is insignificant if likelihood ratio tests indicated variables should be added?

I've been estimating lots of versions of the same model by incrementally adding variables. With some variables, if I add them to the model, the likelihood ratio test indicates that they are ...
user avatar
3 votes
0 answers
175 views

Is BIC useful for a multimodal distribution?

Given a dataset $D$ and a model $M$ with parameters $\theta$, the Bayesian Information Criterion can be used to approximate the model's marginal likelihood $\int p(D|\theta,M)p(\theta|M) d\,\theta$. ...
user avatar
3 votes
0 answers
318 views

Evaluating the magnitude of AIC and BIC reductions

Reading the interesting post Interpretation of variance in multilevel logistic regression I understood that I cannot compare multilevel logistic models looking at the empty model and at variance ...
user avatar
3 votes
0 answers
609 views

How to calculate BIC for multidimensional problem

Sorry for this question, but I am really not sure how to calculate BIC for my situation. My models are mixtures of normals with different number of components. Variances are equal for all components ...
user avatar
3 votes
0 answers
227 views

Bayesian justification for AIC/BIC

Can someone point me to a straightforward and comprehensible Bayesian discussion justifying AIC and/or BIC? Or even better, can someone give a self-contained such discussion in this forum?
user avatar
3 votes
0 answers
350 views

BIC vs. Out of sample performance

I have two statistical models. Model 1 uses a GLM approach while model 2 uses a time series approach for fitting. I want to compare these two models. Model 1 (i.e. GLM) has a better out of sample ...
user avatar
  • 7,158
2 votes
0 answers
30 views

Is it circular reasoning to compute the ELBO using MCMC?

Let's say we have a posterior distribution $q(\theta) = p(\theta \mid D, \mathcal{M})$ over parameters $\theta$ given data $D$ and a model $\mathcal{M}$. As is often the case, computing $q$ is hard, ...
user avatar
  • 21
2 votes
0 answers
29 views

AIC vs BIC for time series clustering and descriptive purposes

I'm in the process of fitting a hidden markov model with gaussian mixtures to time series health data. The primary purpose of this is descriptive, not predictive – I'm using the fitted model to give a ...
user avatar
  • 21
2 votes
0 answers
605 views

Time series analysis VAR model: AIC and BIC test criteria

Consider two variables. Imagine you want to analyse the effects of the lags of variable A on variable B. The possiblity you see an effect of variable A on B is reasonable, but there is absolutely no ...
user avatar
  • 260
2 votes
1 answer
85 views

Linear correlation with number of latent classes and BIC?

I have a dataset with almost 300 cases and 8 variable. Some are binary, some are ordinal variables with 4 to 5 categories. I am performing a hierarchial cluster analysis and latent class analysis in R....
user avatar
  • 725
2 votes
0 answers
57 views

What does it mean to use BIC or AIC to clean residuals?

When I am studying the Augmented Dickey Fuller test I encountered with this phrase in my professor's textbook. It says exactly " In practice we use BIC or AIC to clean the residuals." It made no sense ...
user avatar
2 votes
0 answers
242 views

normalizing Bayesian Network score

I have been doing data driven causal inference from Bayesian Networks in R using bnlearn package on a random categorical dataset....
user avatar
2 votes
0 answers
600 views

Why is BIC overfitting for a large data set but not from a smaller sample?

I'm trying to use the BIC to select between two non-linear models. One model has 2 parameters, the other has 4. The models are based on rate equations for consecutive reactions, and have good support ...
user avatar
2 votes
0 answers
28 views

Mixture of Binomials: Penalization

I have a set of data points generated from 2 random variables: 1st: with prior probability 1 the point was generated by Binomial with 0.5 success probability. 2nd: with priors 0.5 and 0.5 the point ...
user avatar
2 votes
0 answers
93 views

Higher SIC and lower S.E. of residuals

I used the Schwarz Information Criterion (a.k.a., BIC) and the Akaike Information Criterion (AIC) to select the models for a time series Analysis. AIC got me an ARMA (5,4) and SIC got me ARMA (2,1). ...
user avatar
2 votes
0 answers
384 views

When is BIC reasonable approximation to evidence?

I've recently seen a few papers in physics using the Bayesian information criterion (BIC) to evaluate models. I'm much more familiar with Bayesian evidence, $p(x|M)$. I've read in a few places, e.g. ...
user avatar
  • 1,254
2 votes
0 answers
130 views

Random coefficient models with fractional polynomial transformation selection method, which criterion to use?

I am trying to conduct a meta analysis for dose response studies where I am using fractional polynomial transformation from predefined family of powers. Now data fitting using all possible ...
user avatar
  • 31
1 vote
0 answers
44 views

Distribution of Bayesian Information Criteria - using BIC when there are multiple datasets

I have two competing nested models : $M1$ and $M2$. $M1$ has way less parameters. I know that $M2$ is definitely a true model for data (i.e., can explain data fully) but I claim that it is not the ...
user avatar
  • 31
1 vote
0 answers
31 views

Log Likelihood for Two Stage Least Squares (for AIC or BIC)

I'm looking to use a number of information criteria (BIC, AIC, etc.) for Two Stage Least Squares. Of course all information criteria need a log likelihood - and I'm also aware that for a Log-...
user avatar
1 vote
0 answers
104 views

BIC and RMSE are Contradict each other for ARIMA Model Selection using R: Do I Err in Theory or in Practice?

I was surprised to see that RMSE and BIC have contradictory trends for the same time-series data. EDITED The procedures in my code are: simulate a 15 AR series of ...
user avatar
1 vote
1 answer
161 views

Validity of BIC for Dirichlet process mixture models

I am implementing clustering using Dirichlet process mixture models via scikit learn's Variational Bayesian Gaussian Mixture model. I arrived at the appropriate priors iteratively, and I am able to ...
user avatar
  • 63
1 vote
0 answers
28 views

Generalization Error and Matrix Factorizations?

This is more of a discussion/conceptual idea but: Is the notion of generalization error well-defined for a problem such as matrix factorization? I'm working with tensors/matrices and performing ...
user avatar
1 vote
0 answers
103 views

Which GARCH model to choose if both are showing same AIC/BIC and log-likelihood?

I am doing a GARCH model for returns under different error distributions using the R rugarch package. However, two models, under the generalised error distribution and the skewed generalised error ...
user avatar
1 vote
0 answers
37 views

Bayesian Information Criterion (BIC) Applicability

Question Can you compare BIC results between two models that are the same except for the removal of, say, a block of items from one of the models? Situation I'm still in the process of understanding ...
user avatar
1 vote
0 answers
100 views

BIC-Lasso Shrinkage

I am currently reviewing the below paper and was wondering if it was possible to correctly implement the BIC equation for "BIC-LASSO Shrinkage". This doesn't appear to be the same as the typical BIC ...
user avatar
  • 233
1 vote
0 answers
77 views

BIC/AIC Optimal Values

I was reading this paper (https://dms.umontreal.ca/~augusty/FHMV_paper.pdf) and noticed in their analysis (specifically Tables 2 and 3), the highest AIC and BIC values are highlighted and used as ...
user avatar
  • 11
1 vote
0 answers
166 views

AIC / BIC for Model Selection in Copula Model

I'm trying to select the distributional model of 30 marginals (which are restricted to have the same distributional family) in a copula model. However, I therefore get 30 Likelihood/AIC/BIC values for ...
user avatar
  • 73
1 vote
0 answers
1k views

What is the correct implementation of BIC with residual sum of squares?

BIC is most often calculated by maximizing the log likelihood function. However, it is also possible to calculate BIC with residual sums of squares. This is pretty easy to find online and not an issue ...
user avatar
  • 381
1 vote
0 answers
112 views

Calculating BIC based on number of parameters in neural network and loss

I have implemented a variational autoencoder. Now, I would like to calculate the Bayesian Information Criterion (BIC) based on the number of parameters in the network and the loss. For example, let's ...
user avatar
  • 1,524
1 vote
1 answer
285 views

BIC in practice with gaussian distribution

I am considering a Gaussian distribution: \begin{equation} y \sim N(\text{net}(x,w), \sigma^2). \end{equation} where $\text{net}()$ is just the output of some neural net with weights $w$ and input $x$....
user avatar
  • 117
1 vote
0 answers
49 views

Asymptotic equivalence between cross validation and bayesian information criteria

I heard that Bayesian information criteria and cross validation are asymptotically equivalent when the size of validation set is large enough similar to the relationship between Akaike information ...
user avatar
  • 11
1 vote
0 answers
25 views

How to use a previously calculated set of clusters to start EM Clustering in R?

I have performed Hierarchical Clustering on a data set. I would now like to compare the BIC of the clustering methods. The process involves using the clusters you have determined to act as your ...
user avatar
1 vote
0 answers
28 views

AIC and BIC and number of quantization level

I want to test how many quantization levels (discretizing levels) are the best for the given data(time series) set I have. Therefore I am applying different levels of binning (like discretisize data ...
user avatar
  • 21
1 vote
1 answer
257 views

How to choose the number of steps ahead when comparing time series CV to the AIC or the BIC?

I would like to empirically evaluate the performance of the AIC, BIC and Cross Validation as model selection criteria for time series forecasting, i.e. which one of these criteria leads to the best ...
user avatar
  • 10.9k
1 vote
0 answers
1k views

Elbow Method or/and Bayesian Information Criterion to select the optimal number of clusters

I am working k-means cluster analysis. I am new in R and even newer with cluster analysis, but happy to learn. I have managed to plot the following two figures to select the optimal number of ...
user avatar
1 vote
1 answer
145 views

BIC estimate for multidimensional data

I want to use the Bayesian Information Criterium (BIC) to infer the number of clusters in a clustering algorithm, but I have two doubts. Suppose I have $N$ points in a $D$-dimensional space, and that ...
user avatar
1 vote
0 answers
2k views

Correct calculation of BIC (Bayesian Information Criterion) to determine K for K-Means

I am trying to calculate BIC in python. In python, there is no inbuilt library for computing BIC. I referenced the following link to compute variance and BIC further:- Using BIC to estimate the number ...
user avatar
  • 111
1 vote
0 answers
17 views

Bayesian Information Criteria variable penalisation for increasing N

I am trying to understand the Bayesian Information Criteria for model performance, given by the formula below: BIC=−2ln(L)+kln(n) I understand the aim is to minimize this function. I can see that ...
user avatar
1 vote
0 answers
87 views

Interpretation of plots for outlier detection in healthcare

Christy et al. propose cluster-based approach to outlier detection as part of the preprocessing step. However, I don't think the plots are very interpretable. The authors use the R mclustbic function ...
user avatar
  • 11
1 vote
0 answers
2k views

How to calculate BIC and AIC for a gmm model in #R using #plm?

I originally posted this question here. But I think it is better suited for here. I am running Generalized Method of Moments (GMM) Estimation for Panel Data in <...
user avatar
  • 173
1 vote
0 answers
107 views

Goodness of fit + accounting for model complexity + data and predicted values are not on the same scale

I would like to evaluate models based on their goodness of fit with data. Each model produces predicted values that should be correlated to the data but are not necessarily scaled the same way. So far ...
user avatar
  • 11