0
$\begingroup$

PCA performs a linear transformation on a data set to obtain a new data set, this time with eigenvector basis and eigenvalue loadings. If Z is our data set, P our linear transformation, and Y our new data set with eigenvector basis, the transformation is:

P*Z = Y .

but, quiet often Z is not our original data set. If the units of the variables are different, we must perform some sort of standardization/normalization transformation, say K, so that they are comparable. If K is our transformation matrix, then to obtain the new, standardized data set, we perform:

K*X = Z

My question is, what are the loadings in terms of our original data set X? In more practical terms, if b1 is the first measurement type of Z, PC1 might look like..

b1 5.0
b2 2.7
b3 3.0
...
bn 10.0

Where the values of b1->bn are given by P. But what is PC1 in terms of the original data set X, with measurements a1->an?


a1 ?
a2 ?
a3 ?
...
an ?

These should come from some new linear transformation P_new. I would assume we can use basic matrix algebra to figure this out, but I want to make sure I'm 100% correct here:

if K*X = Z,

and P_old*Z = Y, we're searching for:

P_new*X = Y

we should be able to use...

P_new*X = Y

or P_new = Y*X^-1 ?

Is this correct? What if X is not square (and it most certainly is not, there are almost always more trials than variables in any good data set)? Will computational solutions make sense?

I've searched through 23/34 pages on PCA, and was not able to find an answer to this question. If its out there, my apologies, I tried.

Improve this question
$\endgroup$
6
  • $\begingroup$ You might want first to read this thread stats.stackexchange.com/q/62677/3277. $\endgroup$
    – ttnphns
    Commented Oct 9, 2013 at 16:36
  • $\begingroup$ I've read the thread. I'm not asking if performing some pre PCA data-messaging step is theoretically correct, I'm asking how a preprocessing step/transformation effects the final eigenvalue. $\endgroup$
    – tkg
    Commented Oct 9, 2013 at 17:35
  • $\begingroup$ In a linear way, which, however, is unpredictable beforehand. $\endgroup$
    – ttnphns
    Commented Oct 9, 2013 at 17:54
  • $\begingroup$ I'm asking after the fact. Was my question unclear? Here is a simple example. Say I change the units on one of my variables in my data set, this could be seen as a transformation from data set X to data set Z. Then I perform PCA on data set Z, giving eigenvalues and eigenvectors in terms of data set Z. What are these eigenvalues and eigenvectors in terms of data set X? $\endgroup$
    – tkg
    Commented Oct 9, 2013 at 18:41
  • 1
    $\begingroup$ Hmm I don't think thats correct. If K is the linear transformation taking X to Z, and P is the linear transformation taking Z to Y (PCA finds P). In linear algebra's terms, we write KX = Z (K could be normalization or standardization) , and PZ = Y (P is found through PCA). My question was, what is M, the transformation that takes X to Y, written MX = Y. M is found by simple linear algebra! Someone on another forum helped me. Y = PZ = P*(KX) = (PK)*X (matrix multiplication is associative) Hence Y = MX = (PK)*X, which implies MXX^-1 = PKXX^-1 so M = PK ! $\endgroup$
    – tkg
    Commented Oct 10, 2013 at 17:35

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.