I want to fit a mixture of Gaussian to simulated data. Then, I need to calculate the Bayesian information criteria for each mixture component. My point is that, after the model convergence, I calculated the likelihood of each mixture component based on given estimated values. Can that be theoretically correct?.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
It does not make sense to compute a BIC for each component of a mixture. The goal of the BIC is to compute the score of a given model: a mixture, which is a weighted sum of components, is one model. The BIC thus needs to be computed for a whole mixture (i.e. a complete weighted sum of components which is fitted on your whole dataset); computing a BIC for a single component (which has the same number of parameters than all the other components, and is not a model in the sense that it does not explain all your data) does not make sense.
But the BIC can be used to compare different mixtures, for instance to find the optimal number of components to explain your data. Let's assume that the possible range of values for the number of components is $[1,N]$, and you want to estimate the optimal value $n \in [1,N]$ for your dataset. For each $n$ in $[1,N]$, you will fit a mixture with $n$ component(s). Its likelihood will be $\mathcal{L}_n$, and its number of parameters will be $3n-1$ (one mean for each component, one variance for each component, and $n-1$ weights - assuming you don't put any constraints on your components).
Its BIC will thus be $$ BIC_n = -2 \log (\mathcal{L}_n) + (3n-1) \log (T) $$ (assuming that you have $T$ independent data points). The argmax of the above expression for $n$ will give you the optimal number of components to fit your dataset.