Questions tagged [bic]

BIC is an acronym for Bayesian Information Criterion. BIC is one method of model comparison. See also AIC

Filter by
Sorted by
Tagged with
0 votes
0 answers
14 views

Comparing Bayesian hierarchical models with different sample sizes

I have observation data covering a certain period of time. I follow a block-maxima approach where the data are segmented into equal time intervals .My goal is to first develop a Bayesian Hierarchical ...
Ahmed Bayomi's user avatar
4 votes
0 answers
22 views

Why do model selection criteria (xICs, etc) not explicitly incorporate a loss function?

Model Selection and Multimodel Inference by Burnham and Anderson notes that TIC, AIC, AICc and QAICc are based on K-L distance between a given model and true model. Also BIC is in a sense based on ...
Mohan's user avatar
  • 669
9 votes
2 answers
239 views

Is AIC or BIC preferred for prediction/explanation?

This answer (currently 89 upvotes) states: AIC is best for prediction as it is asymptotically equivalent to cross-validation. BIC is best for explanation as it is allows consistent estimation ...
Mohan's user avatar
  • 669
3 votes
1 answer
62 views

BIC not finding a maximum

I was trying to apply BIC to my dataset to find the ideal number of clusters and model that best fits my data, in order to use EM algorithm, but it´s not reaching a maximum, even if I increase the max ...
Inês Pimenta's user avatar
0 votes
1 answer
28 views

How can I rigorously quantify the increase performance due to additional parameters?

I am trying to evaluate a novel dimensionality reduction technique. Specifically, we start with a data set with around 1,000 features/covariates per observation. My technique maps this down to 12. ...
golfWolf's user avatar
0 votes
1 answer
38 views

BIC model comparison [closed]

My go/nogo task was recorded in 4 blocks, each with a new set of cue images, with the learning and therefore the Q-values being reset at the beginning of each block. I have fitted different versions ...
Zahra's user avatar
  • 3
0 votes
0 answers
32 views

BMA formula with BIC

I am interested in using Bayesian modele averaging as a selection creteria (BMA) vs AIC. I read that BMA is widely implemented in clustering models. Suppose that we need to fit M models to a data and ...
Alice's user avatar
  • 640
0 votes
0 answers
23 views

AIC-BIC in mixed models

I was reading about mixed models and I am confused about AIC and BIC criteria. My first question is can I use this types to calculate them? AIC=2d-2ln(l) BIC=dln(n)-2ln(l) where d: is the numbers of ...
Superd's user avatar
  • 31
0 votes
0 answers
32 views

BIC to test good fitting of data to a model

I want to use the Bayesian Information Criterion in order to measure how well a gaussian and 0 order polynomial fit (using python), the one with the lowest BIC should then be the 'best fit' ? My ...
Michael's user avatar
6 votes
1 answer
312 views

Does information criteria (AIC, BIC and DIC...) imply "causality"?

I am interested in finding out the graphical causal structure. Causal Discovery algorithms (e.g., DAG learning) are used to identify potential causal graphs. In score-based causal discovery methods, ...
Jay's user avatar
  • 61
0 votes
1 answer
48 views

Maximum likelihood estimation with unknown distribution

I want to fit a model to some data using structural time series modeling and Kalman filtering. The hyperparameters need to be tuned using maximum likelihood estimation. I am using the Python package ...
eork's user avatar
  • 53
1 vote
0 answers
61 views

Which evaluation metric should I choose? AIC or MSE?

I am currently at a total loss, so I hope someone can point me in the right direction regarding my model selection. The situation I want to create a linear model that best forecasts my data. I am ...
eork's user avatar
  • 53
0 votes
0 answers
31 views

Should I calculate a value for the AIC based on a test set or a training set?

I am aware that a question very similar to mine has already been asked here (Should AIC be reported on training or test data?), but some points remain unclear to me. The accepted answer states: On ...
eork's user avatar
  • 53
0 votes
0 answers
63 views

AIC and BIC Manual Calculations Are a Bit Off From Statsmodels Estimates in Python

I ran a multiple regression using statsmodels. I wanted to verify my understanding of calculations for log-likelihood (ll), AIC and BIC. So I attempted to manually calculate the ll, AIC and BIC for ...
GSA's user avatar
  • 141
0 votes
0 answers
26 views

BIC criterion with negative minimized value of the likelihood function

BIC criterion is defined as BIC=$ k \ln(n)-2 \ln(L)$ where $k$ is the number of parameters, $n$ represents the number of observations and L is the maximized value of the likelihood function of the ...
Shelley's user avatar
  • 111
1 vote
0 answers
14 views

Does anyone know how to model selection for function on function linear model to find the best subset of functional covariate in R [closed]

Does anyone know how to model selection for function on function linear model to find the best subset of functional covariate in R
Sky Sha's user avatar
  • 11
1 vote
1 answer
95 views

BIC drop in regsubset summary different from manual calculation in R

The leaps library regsubset function gives an object that contains the list of BIC drops of each subset model from the intercept model. However, it is different from what is calculated manually. For ...
AlgoManiac's user avatar
0 votes
0 answers
12 views

Model Comparison for Overlapping Models (Choosing between the different measurements of the same skill)

I have a large dataset with 15 independent variables. I am interested in investigating the potential interactions between certain independent variables; however, some independent variables measure the ...
Dennis's user avatar
  • 1
0 votes
0 answers
54 views

Exact computation of Bayes factor for multivariate normal

Question: Is there a known, exact expression for the Bayes factor between two multivariate normal hypotheses? Let $H_1$ and $H_2$ be two subsets of $R^d$ with normal priors $\pi(\mu|H_j)$. The sets $...
tims's user avatar
  • 1
1 vote
1 answer
242 views

How to report on model selection

I have run model selection and have been advised to use the BIC value to determine the best model for my purposes. However, having looked around online I cannot seem to find a standard way to report ...
Rstudioamateur's user avatar
0 votes
0 answers
19 views

How to decide if separate groups of data are best explained by one or multiple models

I have run three separate experiments (A, B, and C) that are mostly the same except for one feature. I have two response variables that I am wanting to understand the relationship between. I want to ...
kdoelling's user avatar
4 votes
3 answers
677 views

Why would a smaller AIC than BIC lead to an increased chance of both overfitting and underfitting?

I am puzzled by the following statement in my lecture notes AIC penalty smaller than BIC; increased chance of overfitting BIC penalty bigger than AIC; increased chance of underfitting Is there a ...
Kirsten's user avatar
  • 703
1 vote
0 answers
23 views

Information criterion for cases where $n\gg k$?

I'm looking for an information criterion metric that effectively penalizes the number of parameters ($k$) in cases where we have huge sample sizes. In my particular case my sample size is $\approx80,...
Haliaetus's user avatar
  • 123
3 votes
1 answer
440 views

Why is BIC considered consistent (though AIC is mostly used) for large number of observation?

AIC, BIC are the famous criteria for model selection. But many times they show different results. I read in several places that BIC is consistent while AIC is not. And AIC can achieve minimax rate but ...
user avatar
0 votes
0 answers
51 views

To the likelihood of what does the loglikelihood in AIC, BIC refer?

Both Akaike's information criterion and Bayesian information criterion are calculated from the loglikelihood, eg AIC= -2 log-likelihood +2k. To the likelihood of what does it refer to? The likelihood ...
qcabepsilon's user avatar
0 votes
0 answers
62 views

What measure to use for comparison of non-linear models with very small number of data points

My question is on the details and subtleties of comparing models in non-linear regression. Situation: I want to find out which function fits a set of data points better. Therefore, I'm looking for ...
qcabepsilon's user avatar
1 vote
1 answer
359 views

How does the Bayesian Information Criterion work for model selection?

I am aware that we can use the BIC values from different models in order to determine which model predicts the data best. However, I'm a little confused about the criteria used to determine which ...
john connor's user avatar
0 votes
1 answer
55 views

Modelling predictor as quadratic term lowers BIC but marginal effect plot shows linear distribution?

I am using a modified Hosmer Lemeshaw approach, culminating in all possible combinations model selection, to conduct multiple logistic regression that should distinguish between use (1) and available ...
SageR's user avatar
  • 25
0 votes
0 answers
140 views

AIC and SC tend to select the maximum lag for VECM

I am building a VAR modell in order to discover how oil price shocks and 3 macroeconomic control variables (gdp_growth, Interest rate, exchange rate) influence core and headline inflation in the USA. ...
Fritz Müller's user avatar
0 votes
0 answers
77 views

BIC values in LCA models

I have been reading posts on this forum about how to assess LCA models. One question asked about BIC statistics to assess model fit. The answer said that you shouldn't really worry about the absolute ...
May Smith's user avatar
0 votes
1 answer
30 views

Comparison problem: GAS (Generalized Autoregressive Score) and GARCH

My goal is to show that t-GAS is better than t-GARCH, so I am trying to analyze some data about Crude Oil volatility comparing these models with functions in R packages (...
Ily Del's user avatar
0 votes
0 answers
20 views

why in BIC Bayesian information criterion we ignore items independent of N [duplicate]

I wonder why BIC is defined in a way where items independent of N are dropped. In other words, why in the following image those in equation 6 that don't vary as N grows are ignored in equation 7. Is ...
Nicolas Fabre's user avatar
1 vote
0 answers
65 views

Comparison of factor structures using AIC and BIC

I tested using a CFA two versions of the same questionnaire: the long version (i.e., 30 items) and the shortened version (i.e., 20 items). It is possible to compare these factor structures using AIC ...
Oliver M.'s user avatar
0 votes
0 answers
149 views

Not able to calculate BIC in bidirectional stepwise regression in R

I am trying to perform stepwise regression with direction as "both" based on BIC in r. But everytime is performing based on AIC only Following is my code ...
jigar somaiya's user avatar
1 vote
1 answer
74 views

n value in the calculation of BIC

This question is building on this post, and on the general formula for BIC. BIC = kln(n) - 2ln(L) I need a bit of help understanding what the term number of ...
Joseph K.'s user avatar
1 vote
1 answer
155 views

BIC for generalized additive models

Is there any way to use BIC in model selection for gam? And if so then how to extract bic?
user avatar
1 vote
2 answers
177 views

Normalising likelihood for BIC/AIC calculation

I am running some model inference using AIC and BIC. My problem is that when I go and calculate the (maximum) loglikelihoods of my models, they are usually really high (range between 4700 and 1400 ...
ltr's user avatar
  • 113
2 votes
1 answer
149 views

How to use priors on the parameter number with an information criterion (AIC, BIC, …)?

Example The example is made up because I hope that it’s more accessible than my actual problem. I want to determine the number of planets of a star. I have: data for some astronomical observable of ...
Wrzlprmft's user avatar
  • 2,261
2 votes
0 answers
89 views

Is it circular reasoning to compute the ELBO using MCMC?

Let's say we have a posterior distribution $q(\theta) = p(\theta \mid D, \mathcal{M})$ over parameters $\theta$ given data $D$ and a model $\mathcal{M}$. As is often the case, computing $q$ is hard, ...
rolu's user avatar
  • 21
0 votes
0 answers
42 views

Difference between Bayesian Information Criteria and Approximate Bayesian Computation as model selection

My question is not very technical and more like a discussion but I will be happy to have a technical input for the comparison b/w BIC and ABC. I am trying to understand and use the best model ...
Usman YousafZai's user avatar
4 votes
1 answer
149 views

Model selection criteria that represent a compromise between AIC and BIC

I am very familiar with the ideas and formula of the two popular model selection criteria AIC/AICc and BIC. When I use them for practical problems in chemometrics, the use of AIC/AICc often gives ...
sshwang's user avatar
  • 43
1 vote
1 answer
350 views

SEM model comparison

I have problem choosing a better measurement model between two (i.e., A and B). Both models have acceptable fit indices. For example, the RMSEA of A is .056; the RMSEA of B is .067. By RMSEA, A seems ...
user359882's user avatar
0 votes
1 answer
477 views

AIC and BIC in GEE

Can I apply Akaike’s information criterion (AIC) and Quasi-likelihood under the independence model information criterion (QIC) in GEE?
Misgan L. Liben's user avatar
2 votes
0 answers
102 views

AIC vs BIC for time series clustering and descriptive purposes

I'm in the process of fitting a hidden markov model with gaussian mixtures to time series health data. The primary purpose of this is descriptive, not predictive – I'm using the fitted model to give a ...
Thomas's user avatar
  • 21
2 votes
2 answers
3k views

AIC, BIC and log likelihood which more important?

I am currently searching for the best ARMA(p,q) model for my conditional mean. When comparing the AIC, BIC and LL, I see that some model perform better in AIC, some in BIC and some in LL. The AC and ...
Elise's user avatar
  • 41
2 votes
0 answers
97 views

Distribution of Bayesian Information Criteria - using BIC when there are multiple datasets

I have two competing nested models : $M1$ and $M2$. $M1$ has way less parameters. I know that $M2$ is definitely a true model for data (i.e., can explain data fully) but I claim that it is not the ...
GSecer's user avatar
  • 41
3 votes
1 answer
392 views

How do you write the AIC and BIC of a regression model in terms of the coefficient-of-determination?

This question is to give a general exposition of the relationship between goodness-of-fit statistics in regression analysis, to answer questions like this one. Consider a nonlinear Gaussian regression ...
Ben's user avatar
  • 118k
2 votes
1 answer
213 views

Linear regression gives a good $R^2$ but also a high BIC

I tried a linear model with interactions on my data ($n=95,840$ rows): ...
Theo75's user avatar
  • 131
10 votes
4 answers
2k views

In what applications do we prefer Model Selection over Model Averaging?

I'm wondering in what applications or scenarios (or in trying to answer what kind of questions), the researcher would prefer using Model Selection (such as AIC or BIC) over Model Averaging (such as ...
ExcitedSnail's user avatar
  • 2,526
0 votes
1 answer
86 views

Regression curve with lowest combined AIC and BIC is a poor predictor

For fun, I was trying to make a predictor for how long it would take for George R. R. Martin's The Winds of Winter to be released. My "best" model is the one that had the lowest combined AIC ...
sangstar's user avatar
  • 131

1
2 3 4 5
7