I'm applying PCA and LDA on my gene expression microarray data to reduce dimensionality. It has ~20,500 features. I tried to visualize the first 3 components from both the methods in a 3d scatter plot and both the methods yielded well separated clusters for the 5 classes of cancer. My goal is to make a classification model, however the 3 components of PCA have explained variance equal to only 0.29 in total. My questions are:
Do the rest of the components merely represent noise or redundancy in data, when 3 components sufficiently segregate it? And so can I use the first 3 PCs or LDs only as my input features for the classification model?
Should I consider the possibility that classes can be segregated in higher dimensions as well? Is it possible to tell? And if so, in such a case, what would be preferable: lower dimensional segregation with less information or higher dimensional segregation with possible redundancy?