Say I have a matrix of feature vectors where I have 1000 samples (feature vectors) while dimension of each feature vector is 10. Can I apply PCA to reduce the dimensions further?
closed as unclear what you're asking by amoeba, John, kjetil b halvorsen, Peter Flom♦ Feb 6 '17 at 13:32
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
What Amoeba and Sycorax said. "Loss of data" is generally interpreted as omitting data points (N getting smaller). Presumably you meant without losing information, which is how Amoeba interpreted it.
If that's the case, the answer to your question is 'it depends'. Consider two extremes: if all of your features are linear multiples of one another then you could reduce it down to one dimension without losing any information. If your features are already orthogonal and the variance is the same in all directions then PCA won't buy you anything.
Consider looking at the eigenvalues of $(X^TX)$. If the eigenvalues drop off sharply then you can possibly drop your dimension further while retaining total variance. If the eigenvalues are roughly equal then PCA won't help you at all.