I have used PCA for dimensionality reduction on a dataset containing mouse EEG containing recordings for each individual mouse. To evaluate the models, I have used a ''leave-one-subject-out'' cross-validation scheme. This means data corresponding to one mouse is left out at every iteration. While doing this, I have computed the principal components based on the data for the remaining mice. These principal components were then used to project the validation data.
Now what I have found is, that the performance score is terrible. I can not figure out why it so. I have computed the statistics (mean, variance, min, max) of the features for each individual and computed both mean and standard deviation for the statistics. They seem to differ by a standard error of multiple times the sample mean. Could this be a reason why PCA failed so terribly ? Can I conclude, that there is a high variance across the individuals with respect to the principal components ?