# How to correctly choose model based on BIC?

I have a question about Bayesian Information Criteria. (GARCH models)

I have looked for so many hours but still very confused about this BIC especially a negative one. As far as I am concerned it is okay to have a BIC that is negative, but the interpretation of them are different in each book on website. I am not looking for sophisticated answer just a normal explanation as if you were to explain someone who is not math or statisticians.

Given same data,length and number of observation which model is better, based on BIC?

1. -4.98749

2. -4.995782

3. -4.9864

I am using R software and running 3 models, GARCH-t, GJR model, and simple GARCH (1,1) model. So, I am trying to see which model is better, based only on BIC. I have already concluded what model is better based on other factors but this makes me confused.

• You should report the software platform that you are using to do this (R, Stata, Ox, ...), because definitions can be different. – tchakravarty Aug 13 '14 at 20:28
• The origin is effectively arbitrary. Smaller BIC (further left on the number line) is better by the criterion. – Glen_b Aug 13 '14 at 21:37
• @Glen_b has already answered your question but it will help if you mention what your data is, how many observations and variables are involved etc. – DataD'oh Sep 2 '17 at 18:39

To at least make sure this has an answer:

The origin is effectively arbitrary. Smaller BIC (further left on the number line) is better by the criterion.

However, one ting to beware of if you compare BICs produced in different ways (using different models, or even the same models produced on different software) is the constants involved in the likelihood; it's common to drop constant terms, but if different models/software don't treat them in an equivalent way the BICs won't be comparable. Sometimes software will tell you exactly what computation is being performed in which case you can usually sort these issues out. When they don't, some detective work may be required. With the same model on different software this is usually easy to spot (and adjust for). With different models you may be able to figure out what is happening if there's a subset of models in common.

In your example, the model with the smallest BIC value, -4.995782, can be taken as the preferred model; but the strength against the other models is nearly non-existent. Typically one would see much larger differences between BIC scores and one model would be clearly preferred; but in your example, the three models are doing an nearly identical job of describing the data.

In fact, with such tiny BIC differences, using other factors to pick the preferred model sounds like exactly the right thing to do.

• But beware of the conservatism in BIC. AIC in most cases is more likely to find the model with the best independently-validated predictive discrimination. – Frank Harrell Oct 8 '17 at 13:03

I can only agree on the issue with tiny differences as this example states. In general, what I can say from my personal experience (mainly academic), BIC differences can be be treated according to the following boundaries:

If the absolute difference $$\delta$$ is greater 10, the smaller BIC value is considerable better.

$$\mid BIC_1-BIC_2\mid > 10$$

If the absolute difference $$\delta$$ is greater 5, the smaller BIC value is likely to be better.

$$\mid BIC_1-BIC_2\mid > 5$$

If the absolute difference $$\delta$$ is smaller 2, the smaller BIC does not indicate to be better than the other.

$$\mid BIC_1-BIC_2\mid < 2$$

Statistics is both, art and science.