I was hoping that someone could simply validate or correct my interpretation of Principal Components Analysis. There are a lot of questions on this site about Principal Components analysis--some listed below. But none of these expressed PCA in the way I refer to below, so I wanted to check if my interpretation was correct or incorrect.

Relationship between SVD and PCA. How to use SVD to perform PCA?

Why are principal components in PCA (eigenvectors of the covariance matrix) mutually orthogonal?

I understand how principal components works as a linear dimensional reduction method, using the spectral decomposition of some matrix $X$. It was wondering if it is appropriate to interpret PCA as simply a change of basis, as we might do in some other situations in linear algebra? So PCA simply takes points expressed in the standard basis and transforms them into points expressed in an eigenvector basis. In this process of transformation, some dimensions with low variance are discarded and hence the resulting dimensional reduction.

Of course the process of finding the eigenvector basis uses numerical optimization, so the resulting projection of the original points into the eigenvector basis will produce some "wiggle" in the points, but otherwise the distances between the points are left unchanged. I am thinking in terms of purely classical PCA here, and not some of the PCA variants where regularization imposes some delibrate properties on the resulting reduced rank projection matrix.


  • 5
    $\begingroup$ Yes. $\phantom{}$ $\endgroup$ – amoeba Nov 16 '18 at 8:35
  • $\begingroup$ @amoeba thanks so much. Yes the validation just helps me to know I am on the right track. $\endgroup$ – krishnab Nov 16 '18 at 16:26
  • $\begingroup$ @amoeba so to clarify -- the distances in the original/standard basis remain approximately unchanged...? Can we say anything about the distances between the points in the transformed/new basis if we kept all the eigenvector basis (e.g. like relative distances remain the same, or something interesting like that)? $\endgroup$ – JPJ Sep 8 '19 at 4:20
  • 1
    $\begingroup$ @JPJ The distances are unaffected by a change of basis. They all remain the same. $\endgroup$ – amoeba Sep 8 '19 at 13:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.