In partial least squares regression, what is the difference between the regression coefficients and the loadings for each independent variable in each component? Specifically, I understand in evety component, each of the independent variables has a coresponding loading. Does each variable also have a regression coefficient? What is the relationship between the loading vector and the coefficients?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
8
Assuming your independent variable matrix is $m\times n$, that you have $m$ observations and $n$ variables.
For each PLS component (AKA latent variable), you get a loading vector ($n \times 1$), so for $h$ components the size of loading matrix ($P$) is $n \times h$. These loadings are calculated for both interpretation and algorithmic purposes but they have no use for prediction.
On the other hand, SIMPLS algorithm (I believe the most popular PLS flavor) also involves calculation of weight matrix ($W$), which has the same size as loading matrix. This orthogonal matrix $W$ is used to calculate $X$ scores ($T$):
$T = X\cdot W$
which is then multiplied by $Y$ loadings ($Q$) for prediction:
$\hat{Y} = T \cdot Q'$
Therefore, the regression coefficients ($\hat{B}$ that is $n\times1$ for a single dependent variable) that can be used to predict $Y$ directly from $X$ can be calculated:
$\hat{B} = W \cdot Q'$
All in all, one obtains a loading vector for each component whereas for different number components a same sized yet different regression coefficients are produced.
As far as I know, a similar logic applies to other PLS algorithms too.
-
$\begingroup$ Thanks for the reply. But I'm still not clear if a regression coefficient is generated for every independent variable - i.e, all $m$ of them, or only a subset. $\endgroup$ Commented May 7, 2019 at 10:06
-
$\begingroup$ Yes, a coefficient for each variable is obtained. But not m but n of them since n is the number of variables. $\endgroup$– gunakkocCommented May 7, 2019 at 10:30
-
$\begingroup$ I suggest re-reading the answer with looking at the matrix/vector sizes carefully, it should help clear things up. $\endgroup$– gunakkocCommented May 7, 2019 at 10:32
-
$\begingroup$ Thanks for the detailed reply. Just one more detail: in your example, what is the dimension of the matrix Q? And Y-hat? I'm asking because I have output from a PLS regression and I have a set of regression coefficients as well as loadings. I get that the coefficients is B-hat and the loading P is n x h. $\endgroup$ Commented May 7, 2019 at 12:30
-
$\begingroup$ I guess Q should be $h$ x 1. Also what is the difference between Q and Q'? $\endgroup$ Commented May 7, 2019 at 12:35