When are latent analyses useful?

As far as I understand, latent profile analysis, clustering or similar latent analyses are about finding something hidden in the data. Are there any guidelines or thoughts on when these techniques are useful? I have seen principal component analysis (PCA) being used for simplifying image data. There it was useful since the learned "structure" could at least be visually verified.

In most other applications, I feel that latent analyses don't provide much information. For example, I could ask a class of 100 students various questions to get, let's say, 6 variables. So, $$n = 100$$ and $$p = 6$$. Now, I could do very sophisticated latent analyses to figure out what lies "underneath" those students.

However, I would say that most latent things that I would find are just random noise. In other words, I could ask the same questions to very many classes and probably I would find 300 different latent profiles, but only a few of them would be theoretically meaningful.

Latent variable analyses, such as factor analysis, are useful when we want to analyze a construct that we can't measure directly in a single question, but which we think MIGHT be imperfectly measured by a whole bunch of different questions. They can be especially helpful if we're not even sure that the thing we want to measure even exists, but we need to find that out.

Here's an example. We think that people might suffer from this thing we are calling "depression," but we don't really know how to measure it, or even if it's just one thing - maybe there are a bunch of different states that we CALL depression but which are really distinct constructs. So how do we proceed? Well we can start by coming up with a list of questions that we think MIGHT measure depression:

Do you often feel sad? Do you have little interest in pleasure or doing things? Do you often feel tired or have little energy? Do you think about hurting yourself? Do you have trouble concentrating?

Of course, some of these questions might not actually measure depression (they might measure anxiety or something else). And some might be better measures than others. But that's what we're going to figure out.

Our theory is that there is some underlying construct "depression" that CAUSES people to give the answers to these questions that they do. If that's true then these variables should all correlate with each other, because they're being influenced by the same thing. If one or more of these variables doesn't strongly correlate with the others, then it's probably NOT being influenced by the same thing as the others (which we assume is depression).

So we throw all of these variables into an exploratory factor analysis. The FA tries to find out of there is one or more underlying latent factor related to all of these items and then it tells us how closely correlated each item is to the underlying factor. Let's say we do this and find that the FA finds only one strong factor, and all of the items "load" pretty strongly on that factor, this strongly suggests that the underlying latent variable is actually "depression." If one item didn't load on it then we would know that item is measuring something else, and could kick it out of the analysis.

Furthermore, since the results of the analysis tell us HOW strongly each item is correlated with the latent variable we can use this information to combine the items together into a new variable that measures depression better than any one item in isolation. We have therefore created a single observed measure of what was previously a unobserved construct that we weren't even sure existed.

This is one of the uses of latent variable analysis.

Another use is if you aren't sure of the structure of the latent variable. For example, if you are interested in "political ideology" but aren't sure if it's just a single "left right" scale, or if there are distinct "economic" and "social" dimensions. To figure this out ask a bunch of questions about both economic and social issues, and throw them all into a factor analysis. Is there just one factor or two? Or three?

(caveat: I'm really only talking about factor analysis or things like latent class analysis here. PCA has a somewhat different logic to it and isn't really designed for latent variable analysis per se from a theoretical perspective, even though you can use it for that. But that's another discussion)

In general, latent variables are some variables we can't directly measure. That is, we cannot directly measure an 8th graders' level of math knowledge, simply because we cant directly connect to their brains and perform this measurement (unlike physical attributes, for example, which we can directly measure). So we provide them with a set of math-related tests - 2 geometry tests, 2 algebra tests, 2 more of word problems. We would like to assume that this set of quizzes properly represents the various aspects of math knowledge for this level (this gets very deep into the area of psychometrics).

Now, assume we have the various scores for each student - we would like to find out the structure of our latent variable, which is math knowledge of 8th graders. There are many methods for this kind of dimensionality reduction, I will relate to the 2 I think are most important:

• PCA: principal component analysis takes a data of $$p$$ columns and returns... $$p$$ columns, each is a linear combination of the original ones. The new variables are promised to be orthogonal to each other, and sorted by proportion of explained variance. Given a threshold (lets say 98% explained variance), we can find what is $$k$$ for which now columns $$1...k$$ explain at least 98% of the variance. The upside: very intuitive, easy to implement, orthogonality. The downside: new variables aren't promised to be interpretable.

• Factor Analysis: Unlike PCA, the FA requires us to provide in advance $$k$$, the number of required factors ($$k, there's a rule of thumb which I can't find right now), then it constructs the appropriate linear combinations. Output variable might or might not be orthogonal to each other (depends on the rotation method selected), but they are almost bound to be interpretable. Moreover, if we construct 2 new variables or 3 new variables, the output variables would differ. If we get back to the collection of math tests, requiring a single factor would probably yield a "general math ability". Requiring 2 factors would yield variables that can be interpreted as "quantitive ability" and "geometry ability", requiring 3 might yield "algebra", "geometry" and "problem-solving" or "math ability", "visual perception" and "reading comprehension". These are just examples. The upside: interpretability, small number of factors. The downside: algorithm is more complex, new variables aren't promised to be orthogonal (which might not be a downside for some).

In this context, I think you should run some exploratory factor analysis on your data to find out if there's anything interesting there. I do think than $$n=100$$ is not a very large dataset. You have namedropped clustering - this is not a method for finding latent variables (at least not in the common meaning) but rather for finding whether the $$n$$ samples could be grouped to some $$g$$ groups.

• Just to be more specific: confirmatory FA requires to specify $k$ in advance, exploratory FA does not. Nov 24 '21 at 20:30
• The answer seems quite generic about what the techniques are and not why they are useful which is what I‘m trying to ask. The first technique is not interpretable, so what‘s the point, I’d say and the second technique is also only moderately interpretable. Specifically, how would we know whether the latent variables that we found have any meaning in the real-world?
– RikH
Nov 24 '21 at 22:13
• You also say that clustering is not a method for finding latent variables. They are definitely similar though (Obersky, 2016).
– RikH
Nov 25 '21 at 9:44
• @Richard, EFA too, most methods, require it to specify k in advance. So, the answer is correct. Nov 25 '21 at 13:30
• @ttnphns, I understand that $k$ is needed in some factor extraction techniques regardless of whether they are going to be used by EFA or CFA. But if we know $k$ ahead of time, it does not sound like it is full EFA but rather CFA or a hybrid of EFA and CFA. After all, EFA is used for finding $k$, among other things. Nov 25 '21 at 13:39