Factor Analysis: Single variable contributing to several latent variables

Question

I was wondering whether factor analysis is right tool in my scenario. That is, I have dataset $X = (X_1, X_2, X_3, X_4)$, where $X_i$ denotes a single variable. As far as I understand factor analysis, it tries to group correlated variables into latent ones. For instance, $(X_1, X_2) \rightarrow Z_1$ and $(X_3, X_4) \rightarrow Z_2$. Here, $Z_i$ denotes the latent variable.

Now let's assume that I am in the possession of domain knowledge and I know that $X_3$ should contribute to $Z_1$ but also to $Z_2$. To adhere with the example above, $(X_1, X_2, \mathbf{X}_3) \rightarrow Z_1$ and $(\mathbf{X}_3, X_4) \rightarrow Z_2$.

Is factor analysis capable of detecting such relationships? When $X_3$ correlates with $X_1$ and $X_2$, and also with $X_4$, wouldn't that imply that at least to some degree $X_1$ and $X_2$ are also correlating with $X_4$? Thus, all four variables would be subsumed into one latent variable, wouldn't they?

Might PCA be more suitable, since features can contribute to multiple PCs?

Answer 1

I assume you are referring to exploratory factor analysis (EFA). To answer your first question, EFA can certainly find whether a variable cross-loads on more than one factor. Choosing whether to retain cross-loading variables in factor analysis is based on both statistical and theoretical decisions. Here is a post on some considerations for whether to retain cross-loading items.

It sounds like you expect an item to cross-load and expect a certain factor structure. In that case, I would recommend using confirmatory factor analysis instead to test the model you are hypothesizing. Please note, that if you go the CFA route, you will need to constrain the factor loadings on Z2 to identify the model because it will only have two indicators (x3 and x4).

I don't think you need to go the PCA route as it sounds like you theoretically interested in factors, not components, and you hypothesize a certain structure.

Edited as I realized I didn't answer your second question. If x1,x2,and x4 correlate with x3, it is possible, though not a given that x1 and x2 will correlate with x4 (think of a Venn Diagram where each circle overlaps with one cirlce (x3) but not each other). If all four do positively correlate, your decision as to whether that means there is one factor or two should be based on theoretical and statistical justifications (theory, parallel test, model fit, etc).