Is it possible to modify the PCA algorithm so that it actually implements factor analysis? We can assume that the uniquenesses are known.

I'm aware that for a $d$-dimensional data $x$, PCA takes the leading $k<d$ eigenvectors of $\Sigma_X$ and uses these to define the principal components.

In factor analysis models this data as $x=Af + \epsilon$ where $f$ is a matrix of factors, $A$ is the loading matrix between the latent factors and the observable data, and $\epsilon$ is the vector of uniquenesses.

I have read (see slide 18 of link) that we should construct $A$ as the leading $k$ eigenvectors of $\Sigma_X - \psi$ where $\psi=\text{diag}(\psi_{11}, \dots , \psi_{dd})$ is the covariance of the $\epsilon$ vector.

Is this true? What is the explanation? The slides are very superficial...

  • 3
    $\begingroup$ The term factor analysis is used in two ways, one that includes techniques such as PCA and a more narrow technical definition that excludes methods such as PCA. If you search on this site for difference between PCA and factor analysis you might find the answers illuminating. $\endgroup$ – ReneBt May 2 at 13:56
  • $\begingroup$ Hi. Thanks for the reply. Yes, I have read several of them but they mostly seem to focus on differences in interpretation and application. I am interested in the specific mathematics of how pca can be adapted to perform factor analysis. I didn't find any threads on that - are you able to offer any insight? Thanks. $\endgroup$ – user11128 May 2 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.