Questions tagged [bic]
BIC is an acronym for Bayesian Information Criterion. BIC is one method of model comparison. See also AIC
263
questions
0
votes
0answers
11 views
Are there assumptions for using the regsubsets function in R?
I'm doing a multiple linear regression with 24 independent variables. I don't need to check the OLS assumptions because I'm only interested in R^2 that each variable explains. To get a smaller model I ...
1
vote
0answers
7 views
Generalization Error and Matrix Factorizations?
This is more of a discussion/conceptual idea but:
Is the notion of generalization error well-defined for a problem such as matrix factorization?
I'm working with tensors/matrices and performing ...
1
vote
0answers
17 views
Which GARCH model to choose if both are showing same AIC/BIC and log-likelihood?
I am doing a GARCH model for returns under different error distributions using the R rugarch package. However, two models, under the generalised error distribution and the skewed generalised error ...
0
votes
0answers
13 views
Calculating SPBIC in R From Lvaan output?
I am starting to believe that my models may not be appropriate for using the normal Bayesian Information Criterion (BIC) to compare the models. I have too many free parameters (ranging from 1000-2000) ...
1
vote
0answers
33 views
Bayesian Information Criterion (BIC) Applicability
Question
Can you compare BIC results between two models that are the same except for the removal of, say, a block of items from one of the models?
Situation
I'm still in the process of understanding ...
0
votes
0answers
15 views
Can we upper bound the KL by BIC or AIC criteria?
I would like to know if it is possible to bound the KL (or the Bayesian free energy a.k.a negative log marginal likelihood) by the Bayesian (and or Akaike) information criterion.
Thank you!
0
votes
0answers
13 views
How to decide if clustering is reasonable (if the number of clusters in a dataset is bigger than one)?
I would like to implement a system that automatically clusters some objects in order to give information whether there is more than one cluster in a specific set of these objects. In other words I ...
0
votes
0answers
8 views
Understanding the false discovery rate and positive selection rate
I want to know the relationship with EBIC and the FDR and PSR of some GLM, and what these two formulas mean?
EBIC is defined as
$$ EBIC(s) = -2\ell(\hat{\beta}(s)) + v(s)log(n)+2v(s)\gamma\...
0
votes
0answers
18 views
How to decide between goodness of fit tests
I have a question on BIC and AIC.
I have a data set and I need to test this data set if it fits various distributions (for example, Gamma or Poisson, etc.) I need to use AIC and BIC statistics for ...
0
votes
0answers
21 views
GMM: Negative BIC values decreasing with k due to small penalty
I am performing GMM clustering on 10 million datapoints with 5 features. I am trying to use the BIC score to estimate the number of clusters, however the BIC score continously decreseases as k ...
0
votes
1answer
24 views
knowing which predictors are significant in a logistic regression model
I am trying to make a logistic regression model based on 5 predictors: 2 of these are categorical and 3 are numerical. The output is simply 1 or 0, and upon performing the Matlab function glmfit(x,y,'...
0
votes
1answer
32 views
Function for bayesian information criterion (BIC)
I am writing my own python function for the bayesian information criterion (BIC) calculation. What I want to do is to choose between two models that I fitted with a set of discrete xy data points.
I ...
0
votes
0answers
19 views
Evaluating goodness of fit of a model estimated with EM-algorithm (with AIC or BIC)
I am learning a Hidden Markov Model with time varying transition probabilities depending on different features. I do this by estimating the model parameters with the EM-algorithm. Now I would like to ...
0
votes
1answer
24 views
Source for BIC with residual sum of squares?
Wikipedia states that under certain circumstances, $\mathrm{BIC} $ can be calculated as:
$$\mathrm{BIC} = n \ln(\mathrm{RSS}/n) + k \ln(n),$$
where $\mathrm{RSS}$ is the residual sum of squares. ...
0
votes
0answers
12 views
Information Criterion vs. Cross Validation for Time Series
I've seen a few posts regarding this question but was hoping for a thorough explanation and possibly a source to go with it. Why would we prefer IC methods over cross-validation for time series data?
...
1
vote
0answers
19 views
BIC-Lasso Shrinkage
I am currently reviewing the below paper and was wondering if it was possible to correctly implement the BIC equation for "BIC-LASSO Shrinkage". This doesn't appear to be the same as the typical BIC ...
0
votes
1answer
75 views
Where is the divide between information criterion (AIC, BIC, etc…) and cross validation?
I've taken a regression class and am now in a machine learning class. In regression, we talk about model selection using adj-R2 and AIC/BIC. In my machine learning class, we primarily select models ...
0
votes
0answers
11 views
Performance of TDA vs likelihood ratio test, AIC, BIC, $R^2$ for model selection
Topological data analysis (TDA) uses topological persistence to find which variables are important and to distinguish signal from noise.
TDA can be used for variable selection.
How does TDA perform ...
0
votes
0answers
16 views
How can I use BIC in R studio where p>n (more predictor variables than sample size)
I am working on a problem that has a sample size of 40, and 200 variables in my data set (Including my Y vector). I am just wondering how I can use R studio to make use of the Bayesian Information ...
2
votes
1answer
40 views
How to perform model selection with the BIC for correlated observations
How is it possible to use the Bayesian Information Criterion (and more generally, to perform model selection) when observations are correlated ?
I would like to compare the BIC of different models ...
1
vote
0answers
26 views
BIC/AIC Optimal Values
I was reading this paper (https://dms.umontreal.ca/~augusty/FHMV_paper.pdf) and noticed in their analysis (specifically Tables 2 and 3), the highest AIC and BIC values are highlighted and used as ...
0
votes
1answer
57 views
Should I compare two models using AIC?
I calculated duration (IV) in seconds using two different ranges (0 to 5s and 0 to 10s). The aim was to find out which range contributes to higher word learning outcomes (dichotomous DV).
I ...
0
votes
0answers
13 views
A dramatic drop in AIC/BIC values (positive to negative) for a certain solution [duplicate]
I was using Stata running latent profile analysis and got fit indices as follows:
AIC 221.25 / -643.82 / 237.25 and BIC 300.94 / -541.36 / 362.47 respectively for 3/4/5-profile solutions.
Is there ...
0
votes
0answers
31 views
Is there an AIC and BIC equivalent for MAP?
I would like to know if there is an AIC or BIC equivalent for maximum a posterior estimation. I'm trying to compare several different models that have been fit using MAP, but I am unsure of the best ...
0
votes
2answers
86 views
Why can't algorithms avoid overfitting themselves?
So, I understand overfitting (bonus question: precise statistical definition of overfitting?). You don't want to match the noise in your sample.
What I don't understand is why this requires a ...
1
vote
1answer
116 views
Using AIC/BIC within cross-validation for likelihood based loss functions
For a course I am teaching, I am having my students fit a Gaussian mixture model using MLEs via the EM algorithm to a bivariate dataset. I have asked the students to use use cross-validation to choose ...
1
vote
1answer
334 views
AIC and BIC values by auto.arima and manual ARIMA
A time series of yearly data, I want to compare the AIC and BIC values by auto.arima and manual ARIMA.
...
0
votes
0answers
21 views
BIC of regression with cutoff
Assume we are fitting a linear regression (or poisson regression) on Y ~ X, and the BIC of this model can be obtained as follow:
...
1
vote
0answers
39 views
AIC / BIC for Model Selection in Copula Model
I'm trying to select the distributional model of 30 marginals (which are restricted to have the same distributional family) in a copula model. However, I therefore get 30 Likelihood/AIC/BIC values for ...
0
votes
0answers
26 views
Interpretation of likelihood function of a continuous random variable in the context of Bayesian Information Criterion
I am confused about the interpretation of the likelihood function of continuously distributed random variable.
It's my understanding that the likelihood function is defined to be
$L(\theta|x) = f_\...
0
votes
0answers
146 views
what is wrong with my Gaussian Mixture density estimation fitting (Python)?
I have a data set (1D) link: dataset, which has values ranging from 21,000 to 8,000,000. When i plot histogram of the log values, i can see there are two peaks, roughly. I tried to fit Gaussian ...
1
vote
0answers
204 views
What is the correct implementation of BIC with residual sum of squares?
BIC is most often calculated by maximizing the log likelihood function. However, it is also possible to calculate BIC with residual sums of squares. This is pretty easy to find online and not an issue ...
0
votes
1answer
79 views
Calculating the AIC based on histograms for selection of stochastic models
I am modelling a nonlinear stochastic process and have data to compare model output against. My aim is to obtain an evolution equation of the form,
$$\frac{du}{dt} = f(u,\theta_f)+\alpha(u,\theta_\...
0
votes
1answer
84 views
Clarification on Akaike's IC (AIC) and BIC for Expectation Maximization with time-changing parameters
I apologize in advance for the trivial question, but I need a clarification on the following issue. Suppose I have a generic model in state-space form described as
$$x_{t+1}=\phi_{t} x_{t}+w_{t+1}$$
$...
1
vote
0answers
54 views
Calculating BIC based on number of parameters in neural network and loss
I have implemented a variational autoencoder. Now, I would like to calculate the Bayesian Information Criterion (BIC) based on the number of parameters in the network and the loss. For example, let's ...
2
votes
2answers
44 views
Test all models possibles is a good manner to choose the “best” model?
I have programmed a function in python to test all possibles linear regression models that I can do with 5 variables. I choose the "best" model in base its AIC and BIC. These models are ...
0
votes
1answer
42 views
Comparing BIC values from variations of a data set
I have a data set of 10000 obs of insurance claims. I've fitted it to a Gamma distribution after dividing my values by 10^8; to a Log-Normal distribution, without changing anything in my data set; and ...
0
votes
0answers
16 views
MLE with higher order markov chain model
I am trying to estimate my Markov chain model. I understand that in a standard model, a higher order will lead to overfitting and higher likelihood. Hence, one can use AIC/BIC to find the order of the ...
18
votes
4answers
680 views
On George Box, Galit Shmueli and the scientific method?
(This question might seem like it is better suited for the Philosophy SE. I am hoping that statisticians can clarify my misconceptions about Box's and Shmueli's statements, hence I am posting it here)....
2
votes
1answer
58 views
Can the BIC be used in neural nets despite its large no. of parameters?
I wonder if I can calculate a BIC value ($BIC=-2\log L(\hat{\theta}) + \log n \cdot d$) for a neural net. The number of parameters is large, $d \approx 20,000$ and I only have $n=700$ samples. Here, ...
0
votes
1answer
91 views
Determining the optimal number of clusters using plots of Bayesian Information Criterion
I am having trouble interpreting the results from an Expectation Maximization clustering using mclust and the Iris flower data, Using R.
Reproducible example
If ...
3
votes
2answers
282 views
Why is the bayesian information criterion called that way?
The word "Bayes" suggests that we are updating a distribution using data, to get a posterior distribution.
The fact that the Bayesian information criterion (BIC) is used to select a model from a set ...
0
votes
1answer
101 views
BIC in practice with gaussian distribution
I am considering a gaussian distribution:
\begin{equation}
y \sim N(net(x,w), \sigma^2).
\end{equation}
$net()$ is just the output of some neural net with weights $w$ and input $x$.
The log-...
3
votes
0answers
76 views
Does a high Chi square p-value for a whole model mean it is insignificant if likelihood ratio tests indicated variables should be added?
I've been estimating lots of versions of the same model by incrementally adding variables. With some variables, if I add them to the model, the likelihood ratio test indicates that they are ...
0
votes
0answers
61 views
Bayesian Information Criterion Formula Proof
while I was digging arima model I saw that BIC value is given as $k*log(n)-2*log(L)$ where $L$ is the maximized value of likelihood function whereas $k$ is number of parameters. I wonder how it is ...
1
vote
1answer
760 views
AIC, BIC values keep changing with lag.max in VAR model
I'm using a VARselect function from vars package in R to select order for my model. My data set has 2 time series with 21 data points.
When I give ...
3
votes
2answers
547 views
3D plot of Akaike Information Criterion (AIC) for suitable ranges of LĖ and k
Giving that Akaike Information Criterion (AIC) is as follow:
How can I Produce a 3D plot of AIC for suitable ranges of LĖ and k.
In other words what could be a suitable ranges of L to try?
Moreover,...
8
votes
1answer
1k views
Is it possible that AIC = BIC?
Two well-known (and related) measures of model complexity from statistics are the Akaike Information Criterion (AIC) and the Bayesian Information
Criterion (BIC).
When might AIC = BIC?
7
votes
2answers
1k views
Logistic regression BIC: what's the right N?
TL;DR: Which $N$ is correct for BIC in logistic regression, the aggregated binomial or Bernoulli $N$?
UPDATES AT BOTTOM
Suppose I have a data set to which I'd like to apply logistic regression. For ...
1
vote
1answer
738 views
Find (or calculate) log-likelihood value, AIC, and BIC for SUR model (for each equation) with systemfit
I have estimated SUR model with systemfit (R package). With the estimated results, I am trying to get logLik, AIC and BIC for ...
