How do we correctly select the final number of classes in latent class analysis?

Question

Hope this is not too basic: What Statistical, Mathematical methods do we use to determine number of

classes and class membership in Latent Class Analysis?

Answer 1

Kathryn Masyn wrote an excellent introduction in 2013 with worked examples of latent class/profile analysis. She covers several model selection indicators, one of them being the Bayesian Information Criterion (BIC), plus two more information-based criteria. You want the smallest value of any of these information criteria. She stated that at the time, no single one was accepted as best.

She also mentions two modified likelihood ratio tests, the bootstrapped LRT (BLRT), and an adjusted likelihood ratio test proposed by Lo, Mendell, and Rubin (LMR LRT). You can't use a regular likelihood ratio test for reasons she explains. These modified LR tests compare models with $K$ versus $K-1$ latent classes; a test p-value <= 0.05 means that you would reject the $K-1$ class model in favor of the $K$ class model. p > 0.05 means both models explain the data about equally well, and because of Occam's Razor (i.e. prefer parsimonious explanations), you would keep the $K-1$ class model.

A 2007 simulation study by Karen Nylund et al (note: this is more technical) revealed that the LMR LRT had higher rates of false-positive results (i.e. it would wrongly select a less parsimonious explanation of the data) than the BLRT. They also found that the BIC was the best of the information criteria that they examined.

However, if you're considering the BLRT over the LMR LRT, note Nylund et al's discussion on page 565. With the BLRT, you're simulating data based on your model results and testing $K$ vs $K-1$ classes (I'm not sure this really qualifies as bootstrapping, but this is what everybody calls it). You're assuming that reality corresponds to the model you are testing, then you simulate data based off that model, and then you obtain the empirical distribution of the difference of -2 * the log likelihood. Now, latent profile analysis typically assumes normally distributed indicators. (Some programs/packages are capable of handling other indicator types, but the same critique would apply). What if that's not reality? What if you assumed normal indicators but they're really counts? What if the empirical distribution of the indicators is really skewed, but isn't well modeled by another distribution either? Then the BLRT results might mislead you. Again, Nylund et al show that when we know that the data generating process corresponds to how we model the indicators in LCA/LPA, the BLRT will outperform the LMR test - but in reality, we do not know the data generating process.

No one method to select the number of latent classes is perfect. Masyn basically said try several. If you have software that can do the BLRT, I'd consider what I said about the distribution of the indicators before I totally hung my hat on that test's result. Latent class analysis is a complex technique. We are also trying to describe a complex reality. All models are wrong. Don't be too hung up on one particular indicator.