Say I have a matrix of feature vectors where I have 1000 samples (feature vectors) while dimension of each feature vector is 10. Can I apply PCA to reduce the dimensions further?

  • 5
    $\begingroup$ What do you mean "without loss of data"? Can you reduce the dimensionality with PCA? Yes you can. Will you lose some information? Yes you will. $\endgroup$
    – amoeba
    Feb 5, 2017 at 23:38
  • 2
    $\begingroup$ @amoeba what if the data lie on a linear sub space? $\endgroup$
    – Sycorax
    Feb 5, 2017 at 23:40

1 Answer 1


What Amoeba and Sycorax said. "Loss of data" is generally interpreted as omitting data points (N getting smaller). Presumably you meant without losing information, which is how Amoeba interpreted it.

If that's the case, the answer to your question is 'it depends'. Consider two extremes: if all of your features are linear multiples of one another then you could reduce it down to one dimension without losing any information. If your features are already orthogonal and the variance is the same in all directions then PCA won't buy you anything.

Consider looking at the eigenvalues of $(X^TX)$. If the eigenvalues drop off sharply then you can possibly drop your dimension further while retaining total variance. If the eigenvalues are roughly equal then PCA won't help you at all.


Not the answer you're looking for? Browse other questions tagged or ask your own question.