I am reading this link PCA

which is a very insightful tutorial

however, in this tutorial, the author mentioned a constraint on PCA:


when we look at eigenvalue/eigenvector definition


there is no requirement for $v^{T}v=I$

I am quite a bit confused here. Can anyone tell me where I got it wrong?

  • $\begingroup$ It's standard; otherwise you'd have an infinite number of eigenvectors for every eigenvalue corresponding to every real number times the eigenvector with norm 1 (i.e., for which $vTv = 1$.) Then no-one would know which eigenvector corresponded to each eigenvalue; to save everyone always having to make that explicit, the standard is to choose the eigenvector for which $v^Tv = 1$. $\endgroup$ – jbowman Mar 19 '18 at 21:32

It's easy to see that eigenvectors can be normalized to any number: $$Av=\lambda v$$ Now multiply both sides by any constant c: $$A(cv)=(cv)\lambda$$

Once you found an eigenvector $v$ you can multiply it by any c and get a vector $cv$ which satisfies the same equation. That's why it's customary to normalize them to 1. This is not the case with eigenvalues. They are unique.

It's convenient in PCA in particular because you often use the operation $s=xv$ where $x$ is the data and $s$ is the score. Normalized to 1 eigenvectors will preserve the norm of $x$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.