Questions tagged [bic]
BIC is an acronym for Bayesian Information Criterion. BIC is one method of model comparison. See also AIC
345
questions
0
votes
0
answers
14
views
Comparing Bayesian hierarchical models with different sample sizes
I have observation data covering a certain period of time. I follow a block-maxima approach where the data are segmented into equal time intervals .My goal is to first develop a Bayesian Hierarchical ...
4
votes
0
answers
22
views
Why do model selection criteria (xICs, etc) not explicitly incorporate a loss function?
Model Selection and Multimodel Inference by Burnham and Anderson notes that TIC, AIC, AICc and QAICc are based on K-L distance between a given model and true model. Also BIC is in a sense based on ...
- 669
9
votes
2
answers
239
views
Is AIC or BIC preferred for prediction/explanation?
This answer (currently 89 upvotes) states:
AIC is best for prediction as it is asymptotically equivalent to cross-validation.
BIC is best for explanation as it is allows consistent estimation ...
- 669
3
votes
1
answer
62
views
BIC not finding a maximum
I was trying to apply BIC to my dataset to find the ideal number of clusters and model that best fits my data, in order to use EM algorithm, but it´s not reaching a maximum, even if I increase the max ...
- 33
0
votes
1
answer
28
views
How can I rigorously quantify the increase performance due to additional parameters?
I am trying to evaluate a novel dimensionality reduction technique. Specifically, we start with a data set with around 1,000 features/covariates per observation. My technique maps this down to 12. ...
- 3
0
votes
1
answer
38
views
BIC model comparison [closed]
My go/nogo task was recorded in 4 blocks, each with a new set of cue images, with the learning and therefore the Q-values being reset at the beginning of each block.
I have fitted different versions ...
- 3
0
votes
0
answers
32
views
BMA formula with BIC
I am interested in using Bayesian modele averaging as a selection creteria (BMA) vs AIC. I read that BMA is widely implemented in clustering models.
Suppose that we need to fit M models to a data and ...
- 640
0
votes
0
answers
23
views
AIC-BIC in mixed models
I was reading about mixed models and I am confused about AIC and BIC criteria. My first question is can I use this types to calculate them?
AIC=2d-2ln(l)
BIC=dln(n)-2ln(l)
where d: is the numbers of ...
- 31
0
votes
0
answers
32
views
BIC to test good fitting of data to a model
I want to use the Bayesian Information Criterion in order to measure how well a gaussian and 0 order polynomial fit (using python), the one with the lowest BIC should then be the 'best fit' ?
My ...
- 1
6
votes
1
answer
312
views
Does information criteria (AIC, BIC and DIC...) imply "causality"?
I am interested in finding out the graphical causal structure. Causal Discovery algorithms (e.g., DAG learning) are used to identify potential causal graphs. In score-based causal discovery methods, ...
- 61
0
votes
1
answer
48
views
Maximum likelihood estimation with unknown distribution
I want to fit a model to some data using structural time series modeling and Kalman filtering. The hyperparameters need to be tuned using maximum likelihood estimation.
I am using the Python package ...
- 53
1
vote
0
answers
61
views
Which evaluation metric should I choose? AIC or MSE?
I am currently at a total loss, so I hope someone can point me in the right direction regarding my model selection.
The situation
I want to create a linear model that best forecasts my data. I am ...
- 53
0
votes
0
answers
31
views
Should I calculate a value for the AIC based on a test set or a training set?
I am aware that a question very similar to mine has already been asked here (Should AIC be reported on training or test data?), but some points remain unclear to me.
The accepted answer states:
On ...
- 53
0
votes
0
answers
63
views
AIC and BIC Manual Calculations Are a Bit Off From Statsmodels Estimates in Python
I ran a multiple regression using statsmodels. I wanted to verify my understanding of calculations for log-likelihood (ll), AIC and BIC. So I attempted to manually calculate the ll, AIC and BIC for ...
- 141
0
votes
0
answers
26
views
BIC criterion with negative minimized value of the likelihood function
BIC criterion is defined as
BIC=$ k \ln(n)-2 \ln(L)$
where $k$ is the number of parameters, $n$ represents the number of observations and L is the maximized value of the likelihood function of the ...
- 111
1
vote
0
answers
14
views
Does anyone know how to model selection for function on function linear model to find the best subset of functional covariate in R [closed]
Does anyone know how to model selection for function on function linear model to find the best subset of functional covariate in R
- 11
1
vote
1
answer
95
views
BIC drop in regsubset summary different from manual calculation in R
The leaps library regsubset function gives an object that contains the list of BIC drops of each subset model from the intercept model.
However, it is different from what is calculated manually.
For ...
- 37
0
votes
0
answers
12
views
Model Comparison for Overlapping Models (Choosing between the different measurements of the same skill)
I have a large dataset with 15 independent variables. I am interested in investigating the potential interactions between certain independent variables; however, some independent variables measure the ...
- 1
0
votes
0
answers
54
views
Exact computation of Bayes factor for multivariate normal
Question: Is there a known, exact expression for the Bayes factor between two multivariate normal hypotheses?
Let $H_1$ and $H_2$ be two subsets of $R^d$ with normal priors $\pi(\mu|H_j)$. The sets $...
- 1
1
vote
1
answer
242
views
How to report on model selection
I have run model selection and have been advised to use the BIC value to determine the best model for my purposes.
However, having looked around online I cannot seem to find a standard way to report ...
0
votes
0
answers
19
views
How to decide if separate groups of data are best explained by one or multiple models
I have run three separate experiments (A, B, and C) that are mostly the same except for one feature. I have two response variables that I am wanting to understand the relationship between. I want to ...
4
votes
3
answers
677
views
Why would a smaller AIC than BIC lead to an increased chance of both overfitting and underfitting?
I am puzzled by the following statement in my lecture notes
AIC penalty smaller than BIC; increased chance of overfitting
BIC penalty bigger than AIC; increased chance of underfitting
Is there a ...
- 703
1
vote
0
answers
23
views
Information criterion for cases where $n\gg k$?
I'm looking for an information criterion metric that effectively penalizes the number of parameters ($k$) in cases where we have huge sample sizes. In my particular case my sample size is $\approx80,...
- 123
3
votes
1
answer
440
views
Why is BIC considered consistent (though AIC is mostly used) for large number of observation?
AIC, BIC are the famous criteria for model selection. But many times they show different results.
I read in several places that BIC is consistent while AIC is not. And AIC can achieve minimax rate but ...
user369390
0
votes
0
answers
51
views
To the likelihood of what does the loglikelihood in AIC, BIC refer?
Both Akaike's information criterion and Bayesian information criterion are calculated from the loglikelihood, eg AIC= -2 log-likelihood +2k.
To the likelihood of what does it refer to? The likelihood ...
0
votes
0
answers
62
views
What measure to use for comparison of non-linear models with very small number of data points
My question is on the details and subtleties of comparing models in non-linear regression.
Situation: I want to find out which function fits a set of data points better. Therefore, I'm looking for ...
1
vote
1
answer
359
views
How does the Bayesian Information Criterion work for model selection?
I am aware that we can use the BIC values from different models in order to determine which model predicts the data best. However, I'm a little confused about the criteria used to determine which ...
- 113
0
votes
1
answer
55
views
Modelling predictor as quadratic term lowers BIC but marginal effect plot shows linear distribution?
I am using a modified Hosmer Lemeshaw approach, culminating in all possible combinations model selection, to conduct multiple logistic regression that should distinguish between use (1) and available ...
- 25
0
votes
0
answers
140
views
AIC and SC tend to select the maximum lag for VECM
I am building a VAR modell in order to discover how oil price shocks and 3 macroeconomic control variables (gdp_growth, Interest rate, exchange rate) influence core and headline inflation in the USA. ...
0
votes
0
answers
77
views
BIC values in LCA models
I have been reading posts on this forum about how to assess LCA models. One question asked about BIC statistics to assess model fit. The answer said that you shouldn't really worry about the absolute ...
0
votes
1
answer
30
views
Comparison problem: GAS (Generalized Autoregressive Score) and GARCH
My goal is to show that t-GAS is better than t-GARCH, so I am trying to analyze some data about Crude Oil volatility comparing these models with functions in R packages (...
- 1
0
votes
0
answers
20
views
why in BIC Bayesian information criterion we ignore items independent of N [duplicate]
I wonder why BIC is defined in a way where items independent of N are dropped. In other words, why in the following image those in equation 6 that don't vary as N grows are ignored in equation 7. Is ...
1
vote
0
answers
65
views
Comparison of factor structures using AIC and BIC
I tested using a CFA two versions of the same questionnaire: the long version (i.e., 30 items) and the shortened version (i.e., 20 items). It is possible to compare these factor structures using AIC ...
- 11
0
votes
0
answers
149
views
Not able to calculate BIC in bidirectional stepwise regression in R
I am trying to perform stepwise regression with direction as "both" based on BIC in r.
But everytime is performing based on AIC only
Following is my code
...
1
vote
1
answer
74
views
n value in the calculation of BIC
This question is building on this post, and on the general formula for BIC.
BIC = kln(n) - 2ln(L)
I need a bit of help understanding what the term number of ...
- 75
1
vote
1
answer
155
views
BIC for generalized additive models
Is there any way to use BIC in model selection for gam? And if so then how to extract bic?
user369390
1
vote
2
answers
177
views
Normalising likelihood for BIC/AIC calculation
I am running some model inference using AIC and BIC. My problem is that when I go and calculate the (maximum) loglikelihoods of my models, they are usually really high (range between 4700 and 1400 ...
- 113
2
votes
1
answer
149
views
How to use priors on the parameter number with an information criterion (AIC, BIC, …)?
Example
The example is made up because I hope that it’s more accessible than my actual problem.
I want to determine the number of planets of a star.
I have:
data for some astronomical observable of ...
- 2,261
2
votes
0
answers
89
views
Is it circular reasoning to compute the ELBO using MCMC?
Let's say we have a posterior distribution $q(\theta) = p(\theta \mid D, \mathcal{M})$ over parameters $\theta$ given data $D$ and a model $\mathcal{M}$. As is often the case, computing $q$ is hard, ...
- 21
0
votes
0
answers
42
views
Difference between Bayesian Information Criteria and Approximate Bayesian Computation as model selection
My question is not very technical and more like a discussion but I will be happy to have a technical input for the comparison b/w BIC and ABC.
I am trying to understand and use the best model ...
- 141
4
votes
1
answer
149
views
Model selection criteria that represent a compromise between AIC and BIC
I am very familiar with the ideas and formula of the two popular model selection criteria AIC/AICc and BIC. When I use them for practical problems in chemometrics, the use of AIC/AICc often gives ...
- 43
1
vote
1
answer
350
views
SEM model comparison
I have problem choosing a better measurement model between two (i.e., A and B). Both models have acceptable fit indices. For example, the RMSEA of A is .056; the RMSEA of B is .067. By RMSEA, A seems ...
- 11
0
votes
1
answer
477
views
AIC and BIC in GEE
Can I apply Akaike’s information criterion (AIC) and Quasi-likelihood under the independence model information criterion (QIC) in GEE?
2
votes
0
answers
102
views
AIC vs BIC for time series clustering and descriptive purposes
I'm in the process of fitting a hidden markov model with gaussian mixtures to time series health data. The primary purpose of this is descriptive, not predictive – I'm using the fitted model to give a ...
- 21
2
votes
2
answers
3k
views
AIC, BIC and log likelihood which more important?
I am currently searching for the best ARMA(p,q) model for my conditional mean.
When comparing the AIC, BIC and LL, I see that some model perform better in AIC, some in BIC and some in LL. The AC and ...
- 41
2
votes
0
answers
97
views
Distribution of Bayesian Information Criteria - using BIC when there are multiple datasets
I have two competing nested models : $M1$ and $M2$.
$M1$ has way less parameters.
I know that $M2$ is definitely a true model for data (i.e., can explain data fully) but I claim that it is not the ...
- 41
3
votes
1
answer
392
views
How do you write the AIC and BIC of a regression model in terms of the coefficient-of-determination?
This question is to give a general exposition of the relationship between goodness-of-fit statistics in regression analysis, to answer questions like this one.
Consider a nonlinear Gaussian regression ...
- 118k
2
votes
1
answer
213
views
Linear regression gives a good $R^2$ but also a high BIC
I tried a linear model with interactions on my data ($n=95,840$ rows):
...
- 131
10
votes
4
answers
2k
views
In what applications do we prefer Model Selection over Model Averaging?
I'm wondering in what applications or scenarios (or in trying to answer what kind of questions), the researcher would prefer using Model Selection (such as AIC or BIC) over Model Averaging (such as ...
- 2,526
0
votes
1
answer
86
views
Regression curve with lowest combined AIC and BIC is a poor predictor
For fun, I was trying to make a predictor for how long it would take for George R. R. Martin's The Winds of Winter to be released. My "best" model is the one that had the lowest combined AIC ...
- 131