I was wondering whether factor analysis is right tool in my scenario.
That is, I have dataset $Y = (X_1, X_2, X_3, X_4)$, where $X_i$ denotes a single variable. As far as I understand factor analysis, it tries to group correlated variables into latent ones. For instance, $(X_1, X_2) \rightarrow Z_1$ and $(X_3, X_4) \rightarrow Z_2$. Here, $Z_i$ denotes the latent variable.

Now let's assume that I am in the possession of domain knowledge and I know that $X_3$ should contribute to $Z_1$ but also to $Z_2$. To adhere with the example above, $(X_1, X_2, \mathbf{X}_3) \rightarrow Z_1$ and $(\mathbf{X}_3, X_4) \rightarrow Z_2$. 

Is factor analysis capable of detecting such relationships? When $X_3$ correlates with $X_1$ and $X_2$, and also with $X_4$, wouldn't that imply that at least to some degree $X_1$ and $X_2$ are also correlating with $X_4$? Thus, all four variables would be subsumed into one latent variable, wouldn't they?

Might PCA be more suitable, since features can contribute to multiple PCs?