When there is only one response (dependent) variable, what is the advantage of partial least squares (PLS) regression over principal component regression (PCR)?

My understanding is that PLS is only better when there are multiple response variables, correct?


They are different methods, independently of the number of response variables. Both methods combine PCA with ordinary multiple regression but it's done in a crucially different way. For a matrix of predictor variables X and one of dependent variables Y, principal component regression performs a PCA on predictor matrix X and then uses those principal components as regressors on Y. This technique removes multicolinearity but does not reduce the number of predictors down to the “best” subset. Picking out manually the most informative components won't work because these components were made from the variables in X and are therefore informative only to X, not Y.

On the other hand, PLS finds components which explain the covariance between X and Y (and calls them “latent vectors”). Hence, with PLS it's safer to assume that more informative components correspond to more relevant predictors.

  • 3
    $\begingroup$ +1. Just to be completely clear in response to the title question: with only one dependent variable, PLS is still different from PCR, they are not identical. $\endgroup$ – amoeba Jul 30 '15 at 13:35
  • $\begingroup$ Yes they're different, even with one dependent variable. I edited my answer to reflect this better. $\endgroup$ – Digio Jul 30 '15 at 13:45
  • 1
    $\begingroup$ I wouldn't say "PLS finds principal components [...]", by the way. "Principal components" is the term reserved for PCA. Perhaps you can say "PLS finds components [...]"? $\endgroup$ – amoeba Jul 30 '15 at 14:15
  • $\begingroup$ I meant "the PLS method", abstractly speaking, but you're right. $\endgroup$ – Digio Jul 30 '15 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.