The first principal component does not separate classes, but other PCs do; how is that possible? I ran PCA on 17 quantitative variables in order to obtain a smaller set of variables, that is principal components, to be used in supervised machine learning for classifying instances into two classes. After PCA the PC1 accounts for 31% of the variance in the data, PC2 accounts for 17%, PC3 accounts for 10%, PC4 accounts for 8%, PC5 accounts for 7% and PC6 accounts for 6%.
However, when I look at mean differences between PCs among the two classes, surprisingly, PC1 is not a good discriminator between the two classes. Remaining PCs are good discriminators. In addition, PC1 becomes irrelevant when used in a decision tree which means that after tree pruning it is not even present in the tree. The tree consists of PC2-PC6.
Is there any explanation for this phenomenon? Can it be something wrong with the derived variables?
 A: I assume that the answer and the example provided by @Flounderer imply this, but I think it worth spelling this out. Principal component analysis (PCA) is label (classification) indifferent. All it does is to transform some high dimensional data to another dimensional space. It might help in classification attempts by, for example, creating data set that is  easier separable by a particular method. However, this is only a by-product (or a side effect) of PCA.
A: It can also happen if the variables are not scaled to have unit variance before doing PCA. For example, for these data (notice that the $y$ scale only goes from $-0.5$ to $1$ whereas $x$ goes from $-3$ to $3$):

PC1 is approximately $x$ and accounts for almost all the variance, but has no discriminatory power, whereas PC2 is $y$ and discriminates perfectly between the classes.
A: When we do Principal Component analysis the principal components correspond to the directions of maximum variability, they do not guarantee maximum discrimination or separation between classes.
So the 2nd component gives you good classification means data in that direction gives you better discrimination between classes. When you perform Linear Discriminant Analysis(LDA) it gives you the best orthogonal direction components that maximize the inter-class distance and minimize the intra-class distance.
So if you do LDA on the data instead of PCA probably one of the very first components would be closer to PC6 than to PC1. Hope this helps.
