0
$\begingroup$

The Bayesian information criterion is defined as $BIC = -2 \text{ln}(L) + k\text{ln}(n)$, where $L$ is the maximized likelihood of the data, and where $n$ is the sample size.

In case of a huge sample size, BIC tend to $\infty$.

Is there any transformation that needs to be done in order to compute the BIC for large samples?

Thank you,

$\endgroup$
1
$\begingroup$

If I got you correctly:

As BIC is basically used to compare models (as AIC or MDL), you can apply any monotone transformation as long as you do it for both of the compared models.

$\endgroup$
4
  • $\begingroup$ Thank you for your post. What transformation would you suggest? $\endgroup$ – Mayou Oct 4 '13 at 13:22
  • $\begingroup$ @Mariam, but just for clarification: what is your confuse confuse about $\mathrm{BIC} \to \infty$? $\endgroup$ – agronskiy Oct 4 '13 at 15:42
  • $\begingroup$ Well I need to compute the BIC for a regression model, where the dependent variable is of length 7000. R returns -Inf for BIC. I am not sure how to compute the information criterion in this case. $\endgroup$ – Mayou Oct 4 '13 at 17:00
  • $\begingroup$ @Mariam Hm... I guess it's rather a computation issue. Try to insert (probably somewhere in the process before you compute BIC) logarithmization (I can't see your computation, but I guess you can reduce the numbers you get by applying something like logarithm). $\endgroup$ – agronskiy Oct 6 '13 at 18:51
0
$\begingroup$

Normalize BIC by dividing By 2n. Let the likelihood function $L = \prod_{i=1}^n p( x_i | \theta)$. Then when the sample size $n$ is large then the influence of the $K \log n$ term becomes negligible and the normalized BIC converges To the cross entropy of the fitted Model with the density which generated the data. To see this note $\frac{BIC}{2n} = -(1/n) \log L+ (k/n) \log (n) = -(1/n) \sum_{i=1}^n \log p( x_i |\theta) + (k/n) \log (n)$ Under usual regularity conditions First normalized log likelihood term on right hand side converges to a number and second term on right hand side converges to zero.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.