# Can Latent Class Analysis entropy be infinity? And what does it mean?

I recently ran a latent class analysis with polytomous outcome variables with the poLCA R package.

The model ran fine converged from 2 to 6 classes (the model converged with more classes but fit indices suggested that fit got worse so I stopped at 6 classes.

While the models converged correctly, it's not able to get the entropy statistic. I normally try to get relative entropy (similar to what MPLus gives) using the code provided in an answer of another question here. It's worked well for me in the past and I was able to get the relative entropy for the models of 2 and 3 classes but gives me a

>NaN


value when I attempt to run it for 4 classes and above. I found this to be odd so I attempted to just get the entropy statistic poLCA gives me with the command

polca.entropy(model)


and that causes Rstudio to just run forever and crash even for 2 and 3 class models that previously got me the relative entropy. What does this mean for the models and is it anything to be concerned about?

I had previously read that BIC is a better measure of model fit for ordered or polytomous outcome variables so I usually never use entropy for anything other than to just report it for the sake of reviewers who are more accustomed to seeing the relative entropy.

Addressing your last paragraph first: Kathryn Masyn, who wrote a chapter about latent class and latent profile models in the Oxford Handbook of quant methods (2013), described entropy as a characteristic, not a model selection criterion. On page 570 of the PDF, she says (emphasis mine):

Because even when $$E_K$$ is close to 1.00 there can be a high degree of latent class assignment error for particular individuals, and because posterior classification uncertainty may increase simply by chance for models with more latent classes, $$E_K$$ was never intended for, nor should it be used for, model selection during the class enumeration process. However, $$E_K$$ values near 0 may indicate that the latent classes are not sufficiently well separated for the $$K$$ classes that have been estimated (Ramaswamy et al., 1993).

Masyn is talking about normalized entropy above, which ranges from 0 to 1. I'm not sure if poLCA gives normalized or non-normalized entropy. Regardless, entropy should not be able to equal infinity.

Normalized entropy is defined in Masyn as:

$$E_K = 1 - \frac{\sum_{i = 1}^n\sum_{k = 1}^K[-\hat p_{ik}ln(\hat p_{ik})]}{n * ln(K)}$$

Where $$n$$ indexes individual observations, $$K$$ indexes all the latent class you identified (e.g. say your final model is 4-classes, $$K=4$$), and $$\hat p_{ik}$$ is the model-estimated probability that the $$i$$-th person belongs to the $$k$$-th latent class (i.e. there are $$K$$ such probabilities for each person).

If entropy is displaying as NaN, I'd suspect some sort of error, and it's not saying the entropy is actually infinity. Maybe it's a programming error (unlikely?). Maybe this is what happens if some observations have some missing data (also unlikely, I think?), or maybe some observations have all indicators missing and this is causing their probabilities to be missing? I'd email the author. No way to provide insight from this end, especially considering that I don't use this package.

• Thank you for your answer! The source you provided was easy to follow. poLCA gives non-normalized entropy, which is the reason why I had to use a separate set of codes to get the relative one that ranges from 0-1.0. It shouldn't be a programming error since it's the same code I used before (and worked) nor should it be due to missing data. I think part of it was that poLCA doesn't handle ordinal variables, only polytomous categorical variables, but my DVs were all ordinal. I ran LPA instead and that gave me the entropy! – ssjjaca Jun 25 at 15:47