How does PCA maximise Total Variance without maximising Co-variance?

https://stats.stackexchange.com/a/3374/92071 - In PCA, the components are actual orthogonal linear combinations that maximize the total variance. In FA, the factors are linear combinations that maximize the shared portion of the variance--underlying "latent constructs".

Now, I understand that eigenvalues represent the amount of variance captured by a particular dimension. In order to obtain these directions, wouldn't one be maximising the co-variance terms along with the variance ones, implicitly?

Varimax rotation (for FA) maximises only the co-variance terms irrespective of the total variance associated with the newly formed dimension. Is this an accurate difference between the two kinds of rotation?

• Varimax or similar rotation of loadings has nothing to do with the difference between PCA and FA: stats.stackexchange.com/q/612/3277 – ttnphns Dec 19 '16 at 8:04
• One, two. PCA accounts for covariance too, because variables co-variate, i.e. share variances. PCA however does not chase after the specific aim to explain the correlatedness as precise as possible. FA does. In FA, the factors are linear combinations Nope, factors are not linear combinations of variables. Factor scores are. – ttnphns Dec 19 '16 at 8:07
• PCA tends to approach, in results, to FA, as the number of variables grows. See thread. The two analyses are nevertheless theoretically distinct. – ttnphns Dec 19 '16 at 8:11
• Yes. FA "takes interest" in off-diagonal elements of the matrix. PCA does not, it however happens to account for them to an extent because covariances are variances shared: explaining total variance you cannot skip explaining shared variance. I recommend you to look through the links I gave. – ttnphns Dec 19 '16 at 9:16
• Varimax rotation (for FA) maximises only the co-variance terms irrespective of the total variance... In this answer I gave an illustrative chart where you can see the essence of PCA-as-rotation along as Varimax rotation. Both are just orthogonal rotations so formally they are comparible. But they optimize different goal. PCA hunts after maximal varince of data points . Varimax seeks to maximize variance of squared loadings of variables for each factor. It doesn't have any aim wrt data cases. – ttnphns Dec 28 '16 at 7:48