2
$\begingroup$

This is the reverse of my previous question Is PCA invariant to orthogonal transformations?

Let $A$ and $B$ be two $n$ x $p$ data matrices where $n$ is the number of samples and $p$ is the number of features.

Both $A$ and $B$ are centred (zero mean for each feature).

The following are the eigenvalue decompositions of their covariance matrices:

$$ \frac{1}{n-1} A^T A = V_A L_A V^T_A $$

$$ \frac{1}{n-1} B^T B = V_B L_B V^T_B $$

Now suppose that $A$ and $B$ have the same principal components, that is:

$$ A V_A = B V_B $$

Does it follow that $B$ is a rotation of $A$ (or vice-versa), that is:

$$ B = AQ \ ? $$

$\endgroup$
3
  • $\begingroup$ I post this question (and the answer below - which I hope is correct) in case it is helpful to someone. It is more or less a theoretical exercise, but it may have a practical application. $\endgroup$ Commented Oct 16, 2016 at 15:49
  • 2
    $\begingroup$ Since $AV_A=BV_B$ is equivalent to $B=A(V_AV^{-1}_B)$ and $Q=V_AV^{-1}_B$ is orthogonal, the answer is evident. $\endgroup$
    – whuber
    Commented Oct 16, 2016 at 17:56
  • $\begingroup$ @whuber indeed, it is, but it wasn't for me at first :) I answered my own question in case someone else needs this. Do you have any thoughts about the second part of my answer? $\endgroup$ Commented Oct 16, 2016 at 18:17

1 Answer 1

2
$\begingroup$

In this equation:

$$ A V_A = B V_B $$

we right-multiply by $ V_B^T $:

$$ A V_A V_B^T = B V_B V_B^T $$

Both $V_A$ and $V_B$ are orthogonal matrices, therefore:

$$ A V_A V_B^T = B $$

Because $ Q = V_A V_B^T $ is a product of two orthogonal matrices, it is therefore an orthogonal matrix. This means that:

$$ AQ = B $$

Note: if two datasets $A$ and $B$ have the same principal components, it could also be that $ B = A T^T $, where $T$ is a translation matrix (which is not orthogonal). However, since data centering is a prerequisite of PCA, $T$ gets ignored.

Also given this post, we can say that: two centred matrices $A$ and $B$ of size $n$ x $p$ are related by an orthogonal transform $ B = AQ $ if and only if their principal components are the same.

$\endgroup$
1
  • $\begingroup$ I did not find the trivial "theorem" in the end nowhere else on the Internet so I thought it would be helpful to write it down somewhere. $\endgroup$ Commented Oct 16, 2016 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.