Under which conditions do PCA and FA yield similar results? Under which conditions can principal components analysis (PCA) and factor analysis (FA) be expected to yield similar results?
 A: This is an excellent question, but unfortunately (or maybe fortunately?) I have only recently written a very long answer in a related thread, addressing your question almost exactly. I would kindly ask you to look there and see if that answers your question.
Very briefly, if we just focus on PCA and FA loadings $\mathbf W$, then the difference is that PCA finds $\mathbf W$ to reconstruct the sample covariance (or correlation) matrix $\mathbf C$ as close as possible: $$\mathbf C \approx \mathbf W \mathbf W^\top,$$ whereas FA finds $\mathbf W$ to reconstruct the off-diagonal part of the covariance (or correlation) matrix only: $$\mathrm{offdiag}\{\mathbf C\} \approx \mathbf W \mathbf W^\top.$$ By this I mean that FA does not care what values $\mathbf W \mathbf W^\top$ has on the diagonal, it only cares about the off-diagonal part.
With this in mind, the answer to your question becomes easy to see. If the number $n$ of variables (size of $\mathbf C$) is large, then the off-diagonal part of $\mathbf C$ is almost the whole matrix (diagonal has size $n$ and the whole matrix size $n^2$, so the contribution of the diagonal is only $1/n \to 0$), and so we can expect that PCA approximates FA well. If the diagonal values are rather small, then again they don't play much role for PCA, and PCA ends up being close to FA, exactly as @ttnphns said above.
If, on the other hand, $\mathbf C$ is either small or strongly dominated by the diagonal (in particular if it has very different values on the diagonal), then PCA will have to bias $\mathbf W$ towards reproducing the diagonal as well, and so will end up being quite different from FA. One example is given in this thread:


*

*Why do PCA and Factor Analysis return different results in this example?
