4
$\begingroup$

I know PCA and PLS are considered as alternative method to each other. But I am thinking about a kind of combination of the two in case of lots of predictors with little variability.

In that case, when I run 1-component PLS with original predictors, it does not produce a meaningful model in terms of prediction. But if I first compute 10-20 PCA components and run 1-component PLS with those PC scores as predictors, practically, the model is quite good in terms of prediction power. But I would like to know why.

Can anybody explain why this is better than 1-component PLS with original predictors?

$\endgroup$
  • 1
    $\begingroup$ @rolando2 "run 1-component PLS" means building a regression model with the first component of PLS analysis. An example of such data would be Redness as dependent and reflectance intensities over 380nm-780nm as predictors. $\endgroup$ – Tae-Sung Shin Apr 5 '12 at 13:04
  • $\begingroup$ Are you trying to reinvent principal component regression by any chance? (en.wikipedia.org/wiki/Principal_component_regression ; function pcr in R package pls) $\endgroup$ – Yevgeny Apr 5 '12 at 13:58
3
$\begingroup$

The reason why 1-component PLS looses to 1-component PLS based on first 10-20 PCA components is that, in the former case, the correlation structure between the original variables is partially lost (since the 1-component PLS direction is chosen only based on the correlations between the original input variables and the outcome variable). In the latter case though all PCA variables are not correlated to each other which makes 1-component PLS equivalent to least squares regression (OLS) based on the PCA variables (i.e. no information is lost apart from loosing the rest of the PCA directions which are not very informative anyway)

In any case, from predictive power perspective, it does not really make sense to limit the number of components in PLS (or PCR) to one - instead the number of components in PLS/PCR should be treated as a "meta" parameter which should be tuned using resampling (e.g. using package caret which provides a really nice harness for that)

$\endgroup$
  • $\begingroup$ Thanks for your answer. Your explanation totally makes sense. If correlations between predictors are zero, PLS = OLS. How did I miss that. :-) $\endgroup$ – Tae-Sung Shin Apr 5 '12 at 17:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.