6
$\begingroup$

At the end of the PCA algorithm one gets a $D\times d$ matrix $U$ such that $z=U^Tx$ (here $x$ is $D$-dimensional and $z$ is $d$ dimensional with $d\leq D$). In multiple sources on the Web I found that $U$ or $U^T$ is named "projection matrix", but according to Wikipedia it cannot be since $U$ is usually not a square matrix (usually $d<D$). The true orthogonal matrix is $UU^T$, which is also named "projection matrix" across the Web.

Maybe it is correct to say that $U$ or $U^T$ is the projection matrix and $UU^T$ is the orthogonal projection matrix?

There is not a clear definition or am I missing something? Are there more appropriate names for those matrices in order to not confuse them?

$\endgroup$
2
  • 2
    $\begingroup$ See stats.stackexchange.com/questions/381991. I've just posted an answer there and it sort of answers this question too. $\endgroup$
    – amoeba
    Dec 17 '18 at 14:56
  • $\begingroup$ @user_anon can you be more exact about the statement that say that U must be a square matrix? Your reference relates to the projection matrix in linear regression which is a different thing. Also how is U not square? PCA is a mapping/projection to spaces of the same dimension. (it is just that you make some columns zero when you reduce the dimensionality) $\endgroup$ Dec 17 '18 at 15:18
2
$\begingroup$

$U U^T$ is the projection operator. I believe calling $z = U^T x$ a projection is an abuse of terminology. It is actually a coordinate transformation, where each value in the new coordinates is computed by a scalar projection onto the principal vectors. $U U^T x$ is the orthogonal projection of the data onto the principal vectors in the original coordinates, and $U^T x$ is the orthogonal projection of the data onto the principal vectors in the new coordinates (principal components).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.