I was wondering whether factor analysis is right tool in my scenario. That is, I have dataset $X = (X_1, X_2, X_3, X_4)$, where $X_i$ denotes a single variable. As far as I understand factor analysis, it tries to group correlated variables into latent ones. For instance, $(X_1, X_2) \rightarrow Z_1$ and $(X_3, X_4) \rightarrow Z_2$. Here, $Z_i$ denotes the latent variable.

Now let's assume that I am in the possession of domain knowledge and I know that $X_3$ should contribute to $Z_1$ but also to $Z_2$. To adhere with the example above, $(X_1, X_2, \mathbf{X}_3) \rightarrow Z_1$ and $(\mathbf{X}_3, X_4) \rightarrow Z_2$.

Is factor analysis capable of detecting such relationships? When $X_3$ correlates with $X_1$ and $X_2$, and also with $X_4$, wouldn't that imply that at least to some degree $X_1$ and $X_2$ are also correlating with $X_4$? Thus, all four variables would be subsumed into one latent variable, wouldn't they?

Might PCA be more suitable, since features can contribute to multiple PCs?

  • $\begingroup$ Thus, all four variables would be subsumed into one latent variable. Not necessary at all. It could be that correlation between X1 or X2 (or even both of them) and X4 is so weak that they hardly at all are loaded by the same single factor. $\endgroup$
    – ttnphns
    Commented Mar 14, 2019 at 18:29
  • $\begingroup$ Factor analysis is not a clustering of variables into exclusive groups. All the m factors extracted load all the p analyzed variables, but they load them differentially, to diverse extent. $\endgroup$
    – ttnphns
    Commented Mar 14, 2019 at 18:32

1 Answer 1


I assume you are referring to exploratory factor analysis (EFA). To answer your first question, EFA can certainly find whether a variable cross-loads on more than one factor. Choosing whether to retain cross-loading variables in factor analysis is based on both statistical and theoretical decisions. Here is a post on some considerations for whether to retain cross-loading items.

It sounds like you expect an item to cross-load and expect a certain factor structure. In that case, I would recommend using confirmatory factor analysis instead to test the model you are hypothesizing. Please note, that if you go the CFA route, you will need to constrain the factor loadings on Z2 to identify the model because it will only have two indicators (x3 and x4).

I don't think you need to go the PCA route as it sounds like you theoretically interested in factors, not components, and you hypothesize a certain structure.

Edited as I realized I didn't answer your second question. If x1,x2,and x4 correlate with x3, it is possible, though not a given that x1 and x2 will correlate with x4 (think of a Venn Diagram where each circle overlaps with one cirlce (x3) but not each other). If all four do positively correlate, your decision as to whether that means there is one factor or two should be based on theoretical and statistical justifications (theory, parallel test, model fit, etc).

  • $\begingroup$ Thanks for the answer. Especially the link was exactly what I was looking for! $\endgroup$ Commented Mar 14, 2019 at 18:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.