1
$\begingroup$

This is the reverse of my previous question Is PCA invariant to orthogonal transformations?

Let $A$ and $B$ be two $n$ x $p$ data matrices where $n$ is the number of samples and $p$ is the number of features.

Both $A$ and $B$ are centred (zero mean for each feature).

The following are the eigenvalue decompositions of their covariance matrices:

$$ \frac{1}{n-1} A^T A = V_A L_A V^T_A $$

$$ \frac{1}{n-1} B^T B = V_B L_B V^T_B $$

Now suppose that $A$ and $B$ have the same principal components, that is:

$$ A V_A = B V_B $$

Does it follow that $B$ is a rotation of $A$ (or vice-versa), that is:

$$ B = AQ \ ? $$

$\endgroup$
  • $\begingroup$ I post this question (and the answer below - which I hope is correct) in case it is helpful to someone. It is more or less a theoretical exercise, but it may have a practical application. $\endgroup$ – turdus-merula Oct 16 '16 at 15:49
  • 2
    $\begingroup$ Since $AV_A=BV_B$ is equivalent to $B=A(V_AV^{-1}_B)$ and $Q=V_AV^{-1}_B$ is orthogonal, the answer is evident. $\endgroup$ – whuber Oct 16 '16 at 17:56
  • $\begingroup$ @whuber indeed, it is, but it wasn't for me at first :) I answered my own question in case someone else needs this. Do you have any thoughts about the second part of my answer? $\endgroup$ – turdus-merula Oct 16 '16 at 18:17
1
$\begingroup$

In this equation:

$$ A V_A = B V_B $$

we right-multiply by $ V_B^T $:

$$ A V_A V_B^T = B V_B V_B^T $$

Both $V_A$ and $V_B$ are orthogonal matrices, therefore:

$$ A V_A V_B^T = B $$

Because $ Q = V_A V_B^T $ is a product of two orthogonal matrices, it is therefore an orthogonal matrix. This means that:

$$ AQ = B $$

Note: if two datasets $A$ and $B$ have the same principal components, it could also be that $ B = A T^T $, where $T$ is a translation matrix (which is not orthogonal). However, since data centering is a prerequisite of PCA, $T$ gets ignored.

Also given this post, we can say that: two centred matrices $A$ and $B$ of size $n$ x $p$ are related by an orthogonal transform $ B = AQ $ if and only if their principal components are the same.

$\endgroup$
  • $\begingroup$ I did not find the trivial "theorem" in the end nowhere else on the Internet so I thought it would be helpful to write it down somewhere. $\endgroup$ – turdus-merula Oct 16 '16 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.