In PCA , consider a 4 x 3 data matrix ( 4 examples each with 3 features ). After getting the 3 eigenvectors (a/b/c) and projecting data on the first 2 vectors, the equation looks like this :

[ first 2 eigen vectors transposed ] x [ data set transposed ] = [ data projected ]

so the first example in the data after being projected would consist of:

$$[a_1x_1 + a_2x_2 + a_3x_3]\\ [b_1x_1 + b_2x_2 + b_3x_3] $$

On recovering data the equation looks like this:

[first 2 eigen vectors] x [projected data] = [original data set]

so that the first example consists of:

$$ a_1[a_1x_1 + a_2x_2 + a_3x_3] + b_1[b_1x_1 + b_2x_2 + b_3x_3]\\ a_2[a_1x_1 + a_2x_2 + a_3x_3] + b_2[b_1x_1 + b_2x_2 + b_3x_3]\\ a_3[a_1x_1 + a_2x_2 + a_3x_3] + b_3[b_1x_1 + b_2x_2 + b_3x_3] $$ My question is how is this equivalent to what the original example looked like $[x_1 x_2 x_3]'$? I know that unit vectors will cancel each other but the equation will be like

$$ a_1[a_1x_1] + b_1[b_1x_1]\\ a_2[a_2x_2] + b_2[b_2x_2]\\ a_3[a_3x_3] + b_3[b_3x_3] $$

giving : $$ x_1 + x_1\\ x_2 + x_2\\ x_3 + x_3 $$


1 Answer 1


If $V$ is the matrix whose columns are the eigenvectors and $x$ is a 3-element column vector, then the transformed vector is given by

$$ x' = V^{T}x $$

To recover the original vector, you would use

$$ x = Vx'= V[V^{T}x] = V[V^{-1}x] = [VV^{-1}]x = x $$

But note that in your example, since you only projected $x$ onto the first two eigenvectors, you can not, in general, recover the original vector. You would need to project $x$ onto the full set of eigenvectors for it to be reversible (assuming there are no redundant eigenvectors).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.