I have the following experience about PCA and I don't understand why. I first do PCA on the original dataset, say, a collection of three-dimensional vectors $(x_1,x_2,x_3)$. The first principal component explains about 99% variance, indicating the dimension of the original dataset is about one. I then drop one dimension (say $x_3$) from the dataset and do PCA (on reduced data $(x_1,x_2)$). This time the first principal component explains only 80% variance.
In PCA, you find new axes that explains the most variance. Your old axes were $x_1,x_2,x_3$; and your new axes (i.e. eigenvectors of covariance matrix) are linear combinations of these axes. For example, the principal component that can explain $99 \%$ of the variance is of the form $e_1=\alpha x_1+\beta x_2+\theta x_3$. If you drop one of the features, say $x_3$, you'll try to explain the data (i.e. $x_1,x_2$) using axes of the form $ax_1+bx_2$, which of course does not guarantee keeping $99 \%$ explainability.