# Factor Analysis: Single variable contributing to several latent variables

I was wondering whether factor analysis is right tool in my scenario. That is, I have dataset $$X = (X_1, X_2, X_3, X_4)$$, where $$X_i$$ denotes a single variable. As far as I understand factor analysis, it tries to group correlated variables into latent ones. For instance, $$(X_1, X_2) \rightarrow Z_1$$ and $$(X_3, X_4) \rightarrow Z_2$$. Here, $$Z_i$$ denotes the latent variable.

Now let's assume that I am in the possession of domain knowledge and I know that $$X_3$$ should contribute to $$Z_1$$ but also to $$Z_2$$. To adhere with the example above, $$(X_1, X_2, \mathbf{X}_3) \rightarrow Z_1$$ and $$(\mathbf{X}_3, X_4) \rightarrow Z_2$$.

Is factor analysis capable of detecting such relationships? When $$X_3$$ correlates with $$X_1$$ and $$X_2$$, and also with $$X_4$$, wouldn't that imply that at least to some degree $$X_1$$ and $$X_2$$ are also correlating with $$X_4$$? Thus, all four variables would be subsumed into one latent variable, wouldn't they?

Might PCA be more suitable, since features can contribute to multiple PCs?

• Thus, all four variables would be subsumed into one latent variable. Not necessary at all. It could be that correlation between X1 or X2 (or even both of them) and X4 is so weak that they hardly at all are loaded by the same single factor. Mar 14 '19 at 18:29
• Factor analysis is not a clustering of variables into exclusive groups. All the m factors extracted load all the p analyzed variables, but they load them differentially, to diverse extent. Mar 14 '19 at 18:32