I have used PCA for dimensionality reduction on a dataset containing mouse EEG containing recordings for each individual mouse. To evaluate the models, I have used a ''leave-one-subject-out'' cross-validation scheme. This means data corresponding to one mouse is left out at every iteration. While doing this, I have computed the principal components based on the data for the remaining mice. These principal components were then used to project the validation data.

Now what I have found is, that the performance score is terrible. I can not figure out why it so. I have computed the statistics (mean, variance, min, max) of the features for each individual and computed both mean and standard deviation for the statistics. They seem to differ by a standard error of multiple times the sample mean. Could this be a reason why PCA failed so terribly ? Can I conclude, that there is a high variance across the individuals with respect to the principal components ?

  • $\begingroup$ What you have not told us is what you want to use these principal components for. For many applications of PCA, such as dimension reduction, there's no good reason that a low cross-validation score should imply a good low-rank representation of the data. $\endgroup$ – Cliff AB Apr 30 '17 at 22:55
  • $\begingroup$ I have used PCA for dimensionality reduction. I want to judge wether the representation found by PCA is realy a good low rank representation. The cross validation result at least indicate, that it is not. $\endgroup$ – Grunwalski May 1 '17 at 11:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.