4
$\begingroup$

I have collected some data and stored them in an $N\times P$ matrix $A$. Using SVD, we can rotate $A=UDV^T$ into a new basis, also discarding some dimensions: $A\approx U\tilde{D}V^T$, where $\tilde{D}$ is a diagonal matrix of the first $k<P$ singular values, with the remaining diagonal values $0$.

Then I train some model using $U\tilde{D}$ as the source data. Now I have a new set of data $B$ that I want to test the model on. The first step in applying the model is rotating $B$ to correspond to the data $U\tilde{D}$ that I trained the model on. How can I do this?

$\endgroup$
  • 1
    $\begingroup$ $UD = UDV^\top V = AV$. So to get your transformed data you need to multiply by $V$. Hence, compute $BV$. $\endgroup$ – amoeba May 3 '16 at 15:00
  • $\begingroup$ Thanks, @amoeba . If you make that an answer, you can accrue upvotes. ;-) $\endgroup$ – Sycorax May 3 '16 at 15:04
  • 1
    $\begingroup$ Which he/she can then cash in at the counter for untold riches. $\endgroup$ – Mark L. Stone May 3 '16 at 15:06
4
$\begingroup$

What you are doing is essentially PCA, even though you did not use this term. It is exactly PCA if your $X$ is centered; if not, then it is a sort of "uncentered PCA".

In any case, to get from $A$ to $UD$ you need to right-multiply it with $V$: $$AV=UDV^\top V = UD.$$

So to apply the same transformation to $B$, you simply right-multiply it with $V$ too.

If you only need the first $k$ columns, then you right-multiply with $V_k$ which is the matrix of the first $k$ columns of $V$.

$\endgroup$
  • $\begingroup$ +1 Thanks! I know this is a basic question, but I was surprised when searching the archives didn't return any results for this. Hopefully your answer will become yet another of your several canonical contributions to CV. $\endgroup$ – Sycorax May 3 '16 at 15:42
  • $\begingroup$ Ah, so you could have self-answered it! $\endgroup$ – amoeba May 3 '16 at 15:44
  • 1
    $\begingroup$ The way I always remember it is that X is n by p then V has to be p by p and so V is what has to be applied to the data to transform the variables. $\endgroup$ – aginensky May 3 '16 at 19:03
  • $\begingroup$ @C11H17N2O2SNa I am wondering if I should interpret you not having accepted this answer as a hint that there is something I should elaborate on? $\endgroup$ – amoeba May 4 '16 at 21:58
  • $\begingroup$ @amoeba I was actually going to place a bounty on it to increase its visibility and hopefully the number of up-votes for everyone. As I said, this question is conspicuously absent in the archives. $\endgroup$ – Sycorax May 4 '16 at 22:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.