3
$\begingroup$

I am using the Latent Class Analysis feature available in Stata 15. The two statistical criterions gave me different indications: $AIC$ suggests me to use 6 classes, instead $BIC$ suggests to use 5 classes. Is this discordance unusual or not? Which is more relaible? Another thing that surprised me is that, according to the values of $AIC$ and $BIC$, the model with 2 classes is preferable to the 3-classes model..but the 4-classes model is preferable to the 2-classes and 3-classes model. I would expect that, if the 4-classes model is the best one, the 3-classes model should be preferable to the 2-classes model. Again, are these results unuasual or not? Thank you for your attention

$\endgroup$
1
$\begingroup$

Apologies that this answer is late, but hopefully it will help someone if not the OP.

Nylund, Asparouhov, and Muthén attempted to shed some light on this question via a simulation study some time ago. They simulated some latent class data with various structures. This is detailed on page 546 and subsequent, but basically they simulated data where the true number of latent classes was 4 or 3. Their table 7 gives their results for AIC, BIC, and one variant of each.

https://www.statmodel.com/download/LCA_tech11_nylund_v83.pdf

I'd especially direct your attention to the results for the 10-item complex LCA - this means that the proportion of each class varied, and that each of the items might have high or low response probabilities in multiple classes (i.e. each item distinguishes more than one latent class, or there isn't a single item that distinguishes any one latent class). I suspect that for many real-world applications, the true structure (if there is one) is probably more like the complex case than the simple case.

In the complex model structure, the BIC was better than AIC at identifying the correct number of classes. The adjusted BIC may have been better than the regular BIC - this statistic has a different adjustment for sample size (see equations on page 545). However, the MPlus forum has some discussion of BIC vs adjusted BIC, and Bengt Muthén said that

I would use BIC. There isn't a solid theoretical theory behind ssa BIC, although some simulation studies have found it to work well.

Linda Muthén said something similar earlier in the thread. The Muthéns are both statisticians and they founded MPlus, which is the specialist's software for this type of analysis. If either of them speaks, I would give that some weight.

TL;DR: BIC > AIC. If you are up for it, you can calculate the sample size adjusted BIC. Stata users can see this link. But see the paragraph above; two knowledgeable statisticians were not that sure that sample size-adjusted BIC was clearly superior to BIC. As to your specific question, going by the simulation study I cited, the AIC has a higher false positive rate than the BIC (i.e. it is more likely to wrongly recommend a higher number of classes than reality). I can't say if your results are unusual, but if the 4-class model has the lowest BIC out of all the models enumerated, then go with that.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.