1
$\begingroup$

I am dealing with quite highly-dimensional data, and am using (in R) Caret's preprocessing 'pca' method to reduce the dimensionality. However, dependent on the number of components I choose, I seem to get quite different (and to me seem inconsistent) results.

Firstly, if I choose to keep 400 principal components:

set.seed(1)
preProcSpectralPC <- preProcess(training[,-1],method = "pca",pcaComp = 400)
trainPC <- predict(preProcSpectralPC,training[,-1])
modelFit <- train(training$age~.,data=trainPC,method = "pls",tuneLength=100)

The optimal number of PCs picked is 39: enter image description here

After I learned this, I decided that I probably didn't need to use 400 PCs, and hence chose to limit the number of PCs from preprocessing to 50:

preProcSpectralPC <- preProcess(training[,-1],method = "pca",pcaComp = 50)

It was my thinking that since 50 principal components is nested within 400, that I would get the same answer for the optimal number of PCs. However, this is not the case, and the number selected is 19:

enter image description here

Does anyone know why I am getting this difference? The only thing I can think is that when it says '#Components', this doesn't necessarily mean the ordered PCs. This could mean that I am keeping let's say 1-19 when I choose 50, but say 1-10 and 60-88 when I choose 400. Is this the reason?

$\endgroup$
0

1 Answer 1

1
$\begingroup$

I think I can answer my own question. The components in the diagrams, model summaries etc. are PLS components not PCA components. When I choose more PCA components, this results in different PLS components. Hence I get different results for 400 vs 50 chosen PCA components.

$\endgroup$
3
  • 1
    $\begingroup$ It is odd to do PCA pre-processing prior to a PLS model. It removes most of the advantages of using PLS since the resulting predictors are uncorrelated. $\endgroup$
    – topepo
    Aug 27, 2015 at 19:17
  • $\begingroup$ @topepo: PCA pre-processing does not only rotate the data to make it uncorrelated, it also reduces dimensionality. $\endgroup$
    – amoeba
    Aug 28, 2015 at 9:02
  • $\begingroup$ Still, PCA-preprocessing before PLS does not make sense IMHO: you need PLS instead of PCA if the variance you're looking for is not in the first PCs, and in that situation it does not make sense to first do PCA and then afterwards try to "rescue" the interesting variance from the PCs you kept. What was your idea behind doing PCA before PLS? $\endgroup$ Aug 28, 2015 at 9:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.