Eigenvectors for correlation and covariance matrix PCA

Question

I know the generally reasons of using correlation matrix vs a covariance matrix when doing PCA (and visa versa) however when thinking about the eigenvectors (principal components of the data) of each of these matrices I still have a question which is:

Obviously, these eigenvectors will be different however since we are just scaling one to get the other is there just a 1:1 mapping to the eigenvectors of a correlation matrix to a covariance matrix? I.e are the principal components we get for one conceptually the same just in different coordinate system or are there total different interpretations for each set of principal components we get when doing PCA with correlation or covariance?

Answer 1

The starting covariance and correlation matrices are a different coordination system and only under very limited circumstance would they bear close resemblance.

In a covariance matrix the variables can have any magnitude of relationship to each other, but in correlation it is constrained so the total variance is 1 and the covariance any fraction of that.

I work mainly with digital signals and they provide evidence that there is no guarantee of correspondence. In the signals I work with the noise level is inversely related to the square root of the intensity. This means for variables with high intensity the covariance matrix picks these out strongly over weak variables. In a correlation matrix the weak variables are given the same weighting even though they are more noisy. Thus a correlation matrix increases noise.

Noise by definition is orthogonal to signal, so it will not be possible to have 1 to 1 correspondence if you have sizeable differences in variable intensity within a signal. If the noise after standardisation is greater than the signal, then your signal disappears from your eigenvectors (or to be precise, it gets split across many noisy eigenvectors based on chance Cross correlations specific to your training set)