# principal component analysis on reduced data

I have the following experience about PCA and I don't understand why. I first do PCA on the original dataset, say, a collection of three-dimensional vectors $$(x_1,x_2,x_3)$$. The first principal component explains about 99% variance, indicating the dimension of the original dataset is about one. I then drop one dimension (say $$x_3$$) from the dataset and do PCA (on reduced data $$(x_1,x_2)$$). This time the first principal component explains only 80% variance.

• What exactly do you drop from the data set?
– Dave
Sep 21 '19 at 19:51
• I have added more details Sep 21 '19 at 19:56
• What were the eigenvalues and eigenvectors? If $x_3$ dominated the first PC, then of course you get a way different result when you drop it!
– Dave
Sep 21 '19 at 20:05
• Could you also check the correlation between the variables? It could be possible that if two variable are correlated strongly, they over emphasize the significance of first principle component. But you really need to check for this, other it could be you're dropping important information and hence PC1 is much smaller. Oct 6 '19 at 17:08

In PCA, you find new axes that explains the most variance. Your old axes were $$x_1,x_2,x_3$$; and your new axes (i.e. eigenvectors of covariance matrix) are linear combinations of these axes. For example, the principal component that can explain $$99 \%$$ of the variance is of the form $$e_1=\alpha x_1+\beta x_2+\theta x_3$$. If you drop one of the features, say $$x_3$$, you'll try to explain the data (i.e. $$x_1,x_2$$) using axes of the form $$ax_1+bx_2$$, which of course does not guarantee keeping $$99 \%$$ explainability.