1
$\begingroup$

We know that PLSR is a very common way to solve Multicollinearity in the Multiple Linear Regression. But do you know how does it work in detail? And why Multicollinearity of $x$ will be related to the multiple dimension of $y$? I can handly find references about this part.

$\endgroup$
2
$\begingroup$

PLSR or partial least squares regression is a dimension reduction technique that shares similarities with principal component analysis.

In principal component regression you seek to obtain a set of new variables (the principal components) that maximize the variance of $X$ and that are uncorrelated to each other.

In PLSR you seek to obtain a set of new variables (the PLS components) that maximize the covariance between $X$ and $y$ and that are uncorrelated to each other.

In both techniques, the new components are uncorrelated. This means that if in your original dataset you were facing a multicolinearity problem (this is, you have predictors in x that are highly correlated between them) by using any of these techniques you will solve the problem, as your components will become uncorrelated.

EDIT: Answer comment

Observe that, in these techniques it is usual to set a threshold on the number of components, so you select the first $k$ components out of a total maximum of $p$ being $p\geq k$

Since PCA maximize the variance of $X$, the first $k$ components are the variables that best explain $X$, but it can happen that, when trying to use these variables in the prediction of $y$, you achieve poor predictive results because the information that related $X$ and $y$ is left in the principal components that you did not select.

On the other hand, PLS maximize covariance between $X$ and $y$. This means that the first $k$ PLS components are the ones that best explain the relation between $X$ and $y$. And for this reason, PLS is expected to provide good predictive results.

Regarding your second question, why Multicollinearity of x will be related to the multiple dimension of y

I am not sure if I am understanding it correctly but I will try to provide an answer. In PLSR, as you say, your response variable can be multidimensional, but this has nothing to do with the multicolinearity of X. It is said that there is a multicolinearity problem if there are variables in x that are highly correlated between them, regardless of having a univariate or multivariate y.

$\endgroup$
8
  • $\begingroup$ Thanks, for your second answer, could I understand as just 1) dim y = 1, use PCA; 2) dim y >1, use PLS? And could you explain why PLS provides better predictive results than PCR? $\endgroup$ – user6703592 Oct 16 '20 at 18:39
  • $\begingroup$ I have answered your comment in the edit section. $\endgroup$ – Álvaro Méndez Civieta Oct 17 '20 at 10:20
  • $\begingroup$ Do you mean even when $dim y > 1,$ we can use PCA to select $k$ components of $x$ independent on $y;$ however we can also use PLS to select $k$ compents of $x$ related to the information of $y?$ By way, if we use PLS and choose first k components of $x,$ should the number of components of $y$ be consistent with $x?$(=k)? Or two numbers are independent chosen? Actually, could you recommend some references of implementation of PLS? $\endgroup$ – user6703592 Oct 17 '20 at 12:41
  • $\begingroup$ Lets say dim(y)=4. You can use PCA (which does not require any knowledge on y) and then use the PCA components to solve 4 regression models (one model per each dimension of y). Or you can use PLS, which is better suited for these purposes.Also, the number of components should be consistent. A good reference for understanding all of this is learnche.org/pid/latent-variable-modelling/index. Chapter 6 deals with PCA, PCR and PLS in detail. $\endgroup$ – Álvaro Méndez Civieta Oct 17 '20 at 13:22
  • $\begingroup$ get it. i will accept your answer. And would you like to answer some questions about the PLS in the further new questions? $\endgroup$ – user6703592 Oct 17 '20 at 14:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.