# How does “Fundamental Theorem of Factor Analysis” apply to PCA, or how are PCA loadings defined?

I'm currently going through a slide set I have for "factor analysis" (PCA as far as I can tell).

In it, the "fundamental theorem of factor analysis" is derived which claims that the correlation matrix of the data going into the analysis ($\bf R$) can be recovered using the matrix of factor loadings ($\bf A$):

$$\bf R = AA^\top$$

This however confuses me. In PCA the matrix of "factor loadings" is given by the matrix of eigenvectors of the covariance/correlation matrix of the data (since we're assuming that the data have been standardized, they are the same), with each eigenvector scaled to have length one. This matrix is orthogonal, thus $\bf AA^\top = I$ which is in general not equal to $\bf R$.

• In addition to @amoeba's answer, look in my answer addresing the terminologic ambiguety. I do not recommend to call eigenvectors matrix A (which are loadings), for reasons of clarity. The (right-side) eigenvector matrix is usually labeled V (because R=USV' by svd), not A. Another equivalent name (coming from biplot terminology) for eigenvectors is "standard coordinates", and for loadings is "principal coordinates". – ttnphns Nov 27 '14 at 18:26
• ("standard coordinates" - because the inertia, or scale of the eigenvalues, is unit magnitude when endowing them; "principal coordinates" - because it is original full magnitude when endowing them.) – ttnphns Nov 27 '14 at 18:31

Indeed, if the eigen-decomposition of the correlation matrix is $$\mathbf R = \mathbf V \mathbf S \mathbf V^\top$$ where $\mathbf V$ are eigenvectors (principal axes) and $\mathbf S$ is a diagonal matrix of eigenvalues, and if we define loadings as $$\mathbf A = \mathbf V \mathbf S^{1/2},$$ then one can easily see that $$\mathbf R = \mathbf A \mathbf A^\top.$$ Moreover, the best rank-$r$ approximation to the correlation matrix is given by the first $r$ PCA loadings: $$\mathbf R \approx \mathbf A_r \mathbf A_r^\top.$$