I understand the main idea of the expression of the AIC and BIC however, I do not understand how they work. I have a small data mtcars from R program. When, I calculate the AIC and BIC, I found that the values of them is high (636 and 660), respectively). I wonder how I got this values while my data is very small. So, does the values of AIC and BIC increase as the data increases? Do they depends on the number of observation? Do I did something wrong?

I did this:

1- Estimate the mixture weights ($\pi$).

2- Estimate the values of the model parameters.

3- Then, calculate the expectation logliklihood function (I am using EM-method).

4- Then, calculate the AIC and BIC.

The logliklihood:

$ll = sum(\tau_1*(log (\pi_1) +log(density_1)) + \tau_2*(log (\pi_2) + log(density_2)))$

$AIC = -2*ll +2k$

$BIC = -2*ll + log(N)*k$

  • 1
    $\begingroup$ The value of AIC and BIC is of no consequence, it is the differences that count. Notice: you have densities in $ll$. Irrespective of the sample size, these densities can be very large or very small depending on the dispersion of the data: if the variance is very small , a gaussian density (which I assume is what you have here) can take an arbitrarily large value close to the mean. $\endgroup$
    – F. Tusell
    Dec 2, 2017 at 6:46

1 Answer 1


The value of AIC and BIC depend on the sample size because the likelihood does (it is a product of a number of terms which depends on the sample size: remember the density you plug in your $ll$ formula is the joint density of the sample). Further, in BIC you have the term log(N) which implies a heavier penalty for parameters in BIC than in AIC if the sample size is large (as log(N) grows larger than 2).

  • $\begingroup$ The sample size is 30. Also, is my ‘ll’ wrong. Or is there any problem with my functions. I will accept the answer. $\endgroup$
    – Alice
    Dec 1, 2017 at 13:59
  • $\begingroup$ No way to tell if your $ll$ is wrong, why should it be? It looks like the likelihood of a mixture. There is nothing suspicious in principle with the values you report, why do you think there is something wrong? $\endgroup$
    – F. Tusell
    Dec 1, 2017 at 16:42
  • $\begingroup$ Yes it is a mixture. I suppose it is wrong since the number is so small hence the AIC AND BIC values must be small. This what I think. $\endgroup$
    – Alice
    Dec 1, 2017 at 19:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.