I have a high dimensional dataset (20k rows, 50 variables). When running PCA on it, PC1 is about 20%, PC2 is 16%, etc. I have to go up to PC30 to get 90% explained variance.
What I'm trying to do is to perform PCA on my data, do k-means clustering, and be able to say PC1 is mostly "gender" related, PC2 is mostly about "income", etc, using the eigenvectors (from my research online, that's how to interpret PCs?).
So what I'm getting is this: low PC1 and PC2s. When I look at the PC1 eigenvalues, the highest value is like .2. When I look at something like PC 24 (which has an extremely low variance explained), I do see one eigenvalue that is .6.
So in terms of interpretability, I'm not sure how to proceed.
- Can I rely on a low eigenvalue on PC1 which has .2 variance explained?
- Can I rely on a fairly high eigenvalue, but with a PC that has a very very low variance explained?