4
$\begingroup$

In Matlab's plsregress function and in many other statistic toolboxes, there is a BETA vector returned that simplyfies the regression problem to(excluding the intercept term in BETA):

Y=X*BETA

In almost all documentations, this BETA vector is used to predict the original data and to calculate residuals from there. This is for the data being used in regression. However, I couldn't find any documentation or method to predict unknowns and using solely the BETA vector for this purpose feels wrong. In other words if I were to calculate Y from a different X what steps should I follow? Is there a clear guide somewhere?

What is the case in cross validation?

Edit: Also in this article http://www.sciencedirect.com/science/article/pii/0003267086800289 the use of BETA is not mentioned for the PLS part.

Months later edit: Now I understand that the source of my confusion is differences between two main algorithms: NIPALS and SIMPLS.

$\endgroup$
11
  • 3
    $\begingroup$ Why does it feel wrong? $\endgroup$
    – amoeba
    May 11, 2016 at 21:12
  • 3
    $\begingroup$ Take PCR. You take your newData and multiply with loadings to get scores of several components. Then you take these scores and multiply with regression weights to get the predicted values. This is mathematically equivalent to taking newData and multiplying it only once by loadings*weights, which is just one beta vector. Does that make sense? $\endgroup$
    – amoeba
    May 11, 2016 at 21:24
  • 2
    $\begingroup$ @whuber: while the answer is that yes, beta is what you need for prediction I think it is a valid question and I'd give it a try answering how to get to the "PLS Beta" from the usual terminology/matrices that is used for PLS in chemometrics. $\endgroup$ May 18, 2016 at 14:31
  • 4
    $\begingroup$ Alternatively, Mevik, B.-H. & Wehrens, R. The pls Package: Principal Component and Partial Least Squares Regression in R, Journal of Statistical Software, 18, 1 - 24 (2007) gives a very nice and concise explanation. (which is of course what @amoeba already said, just a bit more readable as it is not restriced to comment format) $\endgroup$ May 18, 2016 at 14:38
  • 2
    $\begingroup$ @cbeleites: I've re-opened the question then. $\endgroup$ May 21, 2016 at 17:29

1 Answer 1

6
$\begingroup$

I'm mostly using the papers

to extend @amoeba's comment into an answer here:

Let's start with the PLS X

$\mathbf X = \mathbf T \mathbf P' + \mathbf E$ and $\mathbf T = \mathbf X \mathbf W'$

and the Y matrices

$\mathbf Y = \mathbf U \mathbf Q' + \mathbf F$

(outer relations)
(take care to construct the weights $\mathbf W'$ and $\mathbf Q'$ so they refer directly to $\mathbf X$ and $\mathbf Y$, not to deflated matrices!)

Regression can then take place on the X and Y scores: $\hat u = t b$ (inner relation), thus

$\mathbf Y = \mathbf T \mathbf B \mathbf Q' + \mathbf E$

$\mathbf{\hat Y} = \mathbf X \mathbf W' \mathbf B \mathbf Q'$

Now, the last three matrices ($\mathbf W' \mathbf B \mathbf Q'$) are all part of the PLS model parameters. We can therefore introduce one matrix $\mathbf B' = \mathbf W' \mathbf B \mathbf Q'$ which gives PLS coefficients in analogy to the usual MLR coefficients and write
$\mathbf{\hat Y} = \mathbf X \mathbf B'$
which is the usual form of a linear regression model.

Your prediction can either use these "shortcut" coefficients, or the 3 steps of calculating

  1. X scores $\mathbf{\hat T} = \mathbf X \mathbf W'$, then
  2. Y scores $\mathbf{\hat U} = \mathbf{\hat T} \mathbf B$, and finally
  3. $\mathbf{\hat Y} = \mathbf{\hat U} \mathbf Q'$

update: this procedure modeling both $\mathbf X$ and $\mathbf Y$ with latent variables and scores is known as PLS2. In contrast, PLS1 models only one dependent variable $\mathbf y$ (or $\mathbf Y^{(n \times 1)}$ at a time so that no Y-scores are obtained. Multiple dependent variates can be modeled by separate PLS1 models -- one per variate.

Whether multiple PLS1 or a single PLS2 model are better depends on the application, e.g. on whether the dependent variates are correlated and whether an underlying structure with few(er) latent variables is expected.


In practice, you also need to take care of centering (standard practice) and possible scaling (less standard practice) of $\mathbf X$ and $\mathbf Y$.


For cross validation, the prediction works exactly the same way as for unknown cases: you fit the model on your training cases and then predict the left out cases like any other unknown case.

(Assuming this is not asking whether shortcut solutions exist to update a PLS model for exchanging one case during leave-one-out cross validation)

$\endgroup$
6
  • $\begingroup$ Your explanation makes perfect sense. However, in many of the resources such as in eigenvector.com/evriblog/?p=86 the author is regressing the X scores directly to the Y values rather than Y scores. Is this a matter of my decision? Are there any pros or cons? $\endgroup$
    – gunakkoc
    Oct 3, 2016 at 12:07
  • 1
    $\begingroup$ @theGD: the bog entry you link focuses on the $\mathbf X$ side and even though once in a while Barry uses $\mathbf Y$ the formulation uses a a single-column $\mathbf y$ (this is more clearly stated in "The PLS model space revisited" = ref. [4]). A single column doesn't need Y-scores. See also the explanation of the difference between PLS1 (using single column $\mathbf y$ and thus so Y-scores) and PLS2 with Y-scores. With multi-column Y it is your decision whether to build multiple PLS1 models or one PLS2 model. See also e.g. eigenvector.com/faq/index.php?id=93 $\endgroup$ Oct 3, 2016 at 15:43
  • 1
    $\begingroup$ ... and see the update. $\endgroup$ Oct 3, 2016 at 17:47
  • $\begingroup$ The first article you refenced is for NIPALS algorithm while the second article uses SIMPLS. In SIMPLS there is no need for Y-Scores in prediction step (both PLS1 and PLS2) and the equation is simply Y = XRQ' where R is the weights(not to be confused with W in NIPALS) and Q is the Y loadings. The confusion that I had long time ago was due to the differences in these algorithms. Furthermore, the particular function "plsregress" in MATLAB library uses SIMPLS algorithm. $\endgroup$
    – gunakkoc
    Feb 3, 2017 at 13:05
  • $\begingroup$ Thus, can you please update your answer to indicate that your "shortcut" is available for NIPALS and add a little section for SIMPLS (De Jong, S., 1993. SIMPLS: an alternative approach to partial least squares regression. Chemometrics and Intelligent Laboratory Systems) which is more commonly used in many libraries that I have encountered. IMHO, that update will make things more clear for future readers. Lastly, rather than suggesting an update should I answer my own question? $\endgroup$
    – gunakkoc
    Feb 3, 2017 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.