Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
The Bayesian information criterion is defined as $BIC = -2 \text{ln}(L) + k\text{ln}(n)$, where $L$ is the maximized likelihood of the data, and where $n$ is the sample size.
In case of a huge sample size, BIC tend to $\infty$.
Is there any transformation that needs to be done in order to compute the BIC for large samples?
Thank you,
If I got you correctly:
As BIC is basically used to compare models (as AIC or MDL), you can apply any monotone transformation as long as you do it for both of the compared models.
Normalize BIC by dividing By 2n. Let the likelihood function $L = \prod_{i=1}^n p( x_i | \theta)$. Then when the sample size $n$ is large then the influence of the $K \log n$ term becomes negligible and the normalized BIC converges To the cross entropy of the fitted Model with the density which generated the data. To see this note $\frac{BIC}{2n} = -(1/n) \log L+ (k/n) \log (n) = -(1/n) \sum_{i=1}^n \log p( x_i |\theta) + (k/n) \log (n)$ Under usual regularity conditions First normalized log likelihood term on right hand side converges to a number and second term on right hand side converges to zero.
