BIC is an acronym for Bayesian Information Criterion. BIC is one method of model comparison. See also AIC

learn more… | top users | synonyms

0
votes
1answer
28 views

ARMA model selection: in-sample vs. out-sample accuracy measures

I have a time series for 1000 days for many firms. I am interested to know, in general, on what basis I should select an ARMA model (the nature of my problem restricts integration order to 0). Should ...
1
vote
1answer
45 views

ZIP Fit Indices Calculated from an EM Algorithm

I am working through @ben-bolker's owls example available here:https://groups.nceas.ucsb.edu/non-linear-modeling/projects/owls/WRITEUP/owls.pdf In particular, I am making use of the R ...
0
votes
0answers
13 views

range of possible values and model complexity

I have a prediction model where the likelihood is a function of two components $(A,B)$ $L \sim \bf{A}^\alpha + \bf{B}^\beta$ , where both $\bf{A}$ and $\bf{B}$ are $n\times n$ matrices ($n$ is the ...
3
votes
0answers
73 views

Variable selection vs Model selection

So I understand that variable selection is a part of model selection. But what exactly does model selection consist of? Is it more than the following: 1) choose a distribution for your model 2) ...
1
vote
0answers
15 views

Is it meaningful to compare a logit and probit model with the same no. of observations and independent variables using AIC and BIC?

For my dissertation i want to show that the logit and probit model produce the same results. So far i've compared the marginal effects, percent correctly predicted and done a scatter plot of the ...
6
votes
0answers
77 views

Does BIC try to find a true model?

This question is a follow-up or attempt to clear up possible confusion regarding a topic I and many others find a bit difficult, regarding the difference between AIC and BIC. In a very nice answer by ...
0
votes
1answer
34 views

Model-based clustering evaluation with BIC

Let's say I have fitted two models using EM-clustering and they differ in both the number of clusters and are fitted on different subset of features (chosen from the same set of features). Could I ...
0
votes
0answers
19 views

How can a result from reduced mixed model be insignificant (with smaller BIC and AIC) when the whole model gave significant results

Hi I have made a linear mixed model with a number of covariants that could be relevant to control for. This entire model gave me a significant result. However when I reduce the model, removing the ...
1
vote
1answer
39 views

Is it inherently invalid to use BIC for model averaging?

If I understand AIC/AICc vs. BIC correctly, AIC presumes that there is no "true" model and that any given model is simply a "best worst approximation". However, BIC presumes that there IS a "true" ...
0
votes
1answer
26 views

Threshold for AIC for fitting probability distributions to data

I'm fitting probability distributions to my dataset (using Matlab). It is continuous data (later I will probably also face discrete data). I'm ranking the distributions with the corrected AIC score. ...
2
votes
0answers
10 views

Higher SIC and lower S.E. of residuals

I used the Schwarz Information Criterion (a.k.a., BIC) and the Akaike Information Criterion (AIC) to select the models for a time series Analysis. AIC got me an ARMA (5,4) and SIC got me ARMA (2,1). ...
0
votes
1answer
56 views

Bayes Factors for more than 2 hypotheses

Naïve question: I would like to use Bayesian framework for model selection. I have more than 10 models with the same number of parameters (just different assumptions on underlying parameters of the ...
5
votes
2answers
75 views

Why information criterion (not adjusted $R^2$) are used to select appropriate lag order in time series model?

In time series models, like ARMA-GARCH, to select appropriate lag or order of the model different information criterion, like AIC, BIC, SIC etc, are used. My question is very simple, why donot we ...
0
votes
1answer
27 views

Negative BIC in k-means

Probably a simple question but I'm trying to interpret BIC for k-means. I have some k-means clustering and calculating BIC gives me a negative value, with a plot something like this: ...
6
votes
1answer
56 views

Can BIC be Used for Hypothesis Testing

Define the Bayesian information criterion as $$ \mathrm{BIC} = {-2 \cdot \ln{\hat L} + k \cdot (\ln(n) - \ln(2 \pi))} $$ (I do not drop the constant, $ - \ln(2 \pi)$, to avoid issues when equating to ...
3
votes
1answer
53 views

Post Model Selection Inference problems - which remedies exist?

Recently, Hannes Leeb from Yale University and Benedikt Pötscher from the University of Vienna have published a series of papers dealing with what they call Post Model Selection Inference problems.* ...
0
votes
2answers
83 views

AIC/BIC and data transformation

Can you use AIC/BIC to compare models on untransformed data with models on transformed data (such as log, inverse hyperbolic sine, etc.)? I.e. if a model using logged data gives an AIC = 53.62 and a ...
0
votes
0answers
67 views

AIC / BIC alternative in random effect model

I am looking at at panel data set. Hausmann recommends using random effects modeling, I have 3 nested models and Wald believes each step to add information. However, since some of the parameters had ...
0
votes
0answers
66 views

Linear Discriminant Analysis for dimensionality reduction - choosing the dimension

I'm using Linear Discriminant Analysis to do dimensionality reduction of a multi-class data. What is the best method to determine the "correct" number of dimensions? Can I use a method similar to PCA, ...
3
votes
2answers
80 views

Specify order of ARIMA model using autocorrelogram

How can I, using the correlograms above, specify the orders of the ARIMA model? These are the pac an ac of the differenced time series. Using AIC and BIC, I can't seem te find a proper model. ...
0
votes
0answers
24 views

Model comparison with different predictors

I have a conducted a series of experiments and manipulated a variable (X) that - from the literature - I know is relevant in this context. For theoretical reasons, I am now convinced that the ...
0
votes
2answers
53 views

Selecting best ARIMA model with regressors and dummy variable

I have data on GDP, employment rate, inflation and production on two countries and I like to make some ARIMA models. I have done this before, but not with including regressors. Also, the time period ...
1
vote
1answer
53 views

How does one decide the most suitable GARCH model?

I am trying to model financial data using GARCH($p$,$q$). My question is, what information criteria do I use to determine which orders for $p$ and $q$ are most suitable? For instance, for ARIMA ...
3
votes
1answer
48 views

Match model selection strategies with modelling objectives

I am confused trying to match different model selection strategies with different modelling objectives. (Unfortunately, my confusion is reflected in the length of the post. Please be patient.) Model ...
0
votes
0answers
70 views

Zero-inflated Poisson regression Vuong test: Raw, AIC- or BIC-corrected results

I'm analyzing count data for a set of ten species and found that for the five species with highest detection rate, the zero-inflated poisson (ZIP) regression fits the data significantly better than ...
0
votes
1answer
45 views

Criterion for model selection in an AR model?

I would like to smooth some financial time series data under the assumption that the data consists of variable trend and cyclic components plus white noise. I am thinking of applying an AR model to ...
0
votes
0answers
21 views

Calculating 'k' for k-Means and Expectation Maximization

This question inspired my question. I've read a lot of articles on the Internet, and it seems like most people use sums of squares to find 'k' for k-Means and they use BIC to find 'k' for Expectation ...
1
vote
0answers
99 views

F-test vs. BIC for forward stepwise nested linear model selection

I'm an astronomer trying to choose an appropriate linear model for some time series data. The models I'm choosing between are successively higher-order polynomials in the independent variable $t$ ($y ...
0
votes
0answers
75 views

Choose model by BIC in a stepwise algorithm after choosing model from glmnet

I have data where number of observation n is smaller than number of variables p. The answer variable is binary. For example: ...
2
votes
1answer
54 views

Can we apply BIC to select between models in which the parameters are estimated by MCMC

I never considered this problem before. Assume we have two models, $A$ and $B$, we estimate the parameter in $A$ and $B$ respectively by sample mean. Two questions: We can always use the means of ...
3
votes
0answers
242 views

Paradox in model selection (AIC, BIC, to explain or to predict?)

Having read Galit Shmueli's "To Explain or to Predict" (2010) I am puzzled by an apparent contradiction. There are three starting points, AIC- versus BIC-based model choice (end of p. 300 - start of ...
0
votes
1answer
47 views

Bayesian Information Criterion and Basic Marginal likelihood identity

Given the basic marginal likelihood identity: $$ ln \; m(y)=ln \; p(y|\theta^*)+ln\; \pi(\theta^*)-ln \pi(\theta^*|y) $$ is there a way to derive from this the Bayesian Information Criterion? ...
2
votes
1answer
52 views

Effective sample size: does it depend on the model?

When applying the Bayesian information criterion, one has to use an "effective sample size" in the penalty term. E.g. if observing longitudinal data (e.g. changes in the blood pressure of an ...
2
votes
1answer
134 views

How do you derive AIC and BIC for discrete-valued observables?

Let's say I have an experiment which yields discrete results between 1 and $N$. I am modelling the results using a number of statistical models and want to use Akaike (corrected) or Bayesian ...
1
vote
2answers
750 views

X-means algorithm and BIC

I want to simulate X-means algorithm based on [1] in MATLAB. I have some questions about this algorithm. X-means Algorithm Steps: (1) Initialize K = Kmin. (2) Run K-means algorithm. (3) FOR k = ...
1
vote
0answers
60 views

BIC difference for model selection when models have different (and correlated) predictors

I have a binomial dependent variable Y and two main IVs: A is categorical (5 non ordered levels) and B is continuous. A and B are collinear (I tested the effect of A on B with a linear model, and it ...
1
vote
0answers
84 views

X-Means Likelihood for BIC

I have recently been trying to understand the X-means method for deciding on K, using BIC. However I have become stuck on one particular equation in the original paper. On the 4th page, when ...
0
votes
0answers
221 views

Using BIC,AIC for estimating number of clusters in document clustering using Kmeans

In my approach I am trying to find the optimal value of 'k' for clustering a set of documents using KMEANS algorithm. I wanted to use 'AIC' and 'BIC' information criterion function for finding the ...
0
votes
1answer
60 views

Best candidate model using AIC or BIC equal to initial model used to generate simulated data?

For a given ARMA model (order and coefficients are known) we generate simulated data. Model is stationary and invertible. Then using this data, I want to find the best model by trying all combinations ...
0
votes
1answer
51 views

comparing ARIMA and AR with external regressor

Consider the following models fitted to the same time series: ARIMA(0,1,1) ARIMA(1,0,0) (that is, AR(1)) with an external regressor Can I use the AIC (or any other information criteria) to decide ...
1
vote
1answer
75 views

Calculating BIC for a HMM without any training data

I need to evaluate my HMM models, that are trained using EM algorithm, since I don't have any training data. In order to evaluate with BIC or with most of the other criterions I need the log ...
2
votes
2answers
271 views

BIC selection yields much smaller model than AIC - can I use the likelihood ratio test to compare?

I'm trying to model the data (not make predictions) and am NOT using lasso for this, just want to know if my plan is somewhat reasonable here: I'm modelling for a "yes/no" response variable, so I ...
1
vote
0answers
131 views

X-Means Calculation of BIC

I am trying to calculate the BIC for the X-Means algorithm as described in the paper by Pelleg and Moore (https://www.cs.cmu.edu/~dpelleg/download/xmeans.pdf). The paper describes the calculation of ...
0
votes
0answers
306 views

Implementing the Bayesian Information Criterion (BIC) Using PyKalman

I'm trying to use pykalman to do a Kalman filter on financial data and it seems to be generally working very well. However, when I attempt to extend the code using BIC $\mathrm{BIC} = {-2 \cdot ...
1
vote
1answer
87 views

Reasons for EGARCH(1,1) producing higher/worse AIC/BIC than GARCH(1,1)

I am using the log returns of 3 different stock indices. Two of them show improvements in AIC/BIC critereon when I fit EGARCH(1,1) in comparison to GARCH(1,1). One does not. Assuming that estimation ...
1
vote
1answer
154 views

Comparabilty of BIC, AIC and ICL from Mclust. HDclassif and fem objects

I have a question regarding the way BIC, AIC and ICL are computed in the packages mclust, HDclassif and fisherEM. Both of these packages use the negation of AIC, BIC, ICL (bigger is better). This is ...
5
votes
1answer
67 views

BIC in Item Response Theory Models: Using log(N) vs log(N*I) as a weight

In IRT software packages and in the literature it is common to calculate the BIC as $$ \mathrm{BIC} = -2 \cdot \mathrm{logLik} + \log(N)\mathrm{Npars} $$ where $N$ is the number of rows in wide ...
1
vote
1answer
87 views

Modeling different lag structures

I know there are various information criteria that can be used to compare model specifications, including those with different lag structures. I can easily compare the Akaike Information Criterion ...
1
vote
0answers
32 views

BIC for Bayesian ANOVA

I am doing a Bayesian ANOVA as follows: BIC0 = -2 * logLik0 + k0 * log(N) # null hypothesis BIC BIC1 = -2 * logLik1 + k1 * log(N) # alternate hypothesis BIC ...
3
votes
1answer
119 views

Sample size when fitting categorical survey data

I have a model which fits data from repeated surveys: at time $t$, a number $n_t$ respondents is asked a question and can give one of $K$ answers ($k=1, ..., K$). This is repeated $T$ times ($t = 1, ...