2
$\begingroup$

In Matlab's plsregress function and in many other statistic toolboxes, there is a BETA vector returned that simplyfies the regression problem to(excluding the intercept term in BETA):

Y=X*BETA

In almost all documentations, this BETA vector is used to predict the original data and to calculate residuals from there. This is for the data being used in regression. However, I couldn't find any documentation or method to predict unknowns and using solely the BETA vector for this purpose feels wrong. In other words if I were to calculate Y from a different X what steps should I follow? Is there a clear guide somewhere?

What is the case in cross validation?

Edit: Also in this article http://www.sciencedirect.com/science/article/pii/0003267086800289 the use of BETA is not mentioned for the PLS part.

Months later edit: Now I understand that the source of my confusion is differences between two main algorithms: NIPALS and SIMPLS.

$\endgroup$
  • 3
    $\begingroup$ Why does it feel wrong? $\endgroup$ – amoeba says Reinstate Monica May 11 '16 at 21:12
  • 3
    $\begingroup$ Take PCR. You take your newData and multiply with loadings to get scores of several components. Then you take these scores and multiply with regression weights to get the predicted values. This is mathematically equivalent to taking newData and multiplying it only once by loadings*weights, which is just one beta vector. Does that make sense? $\endgroup$ – amoeba says Reinstate Monica May 11 '16 at 21:24
  • 2
    $\begingroup$ @whuber: while the answer is that yes, beta is what you need for prediction I think it is a valid question and I'd give it a try answering how to get to the "PLS Beta" from the usual terminology/matrices that is used for PLS in chemometrics. $\endgroup$ – cbeleites supports Monica May 18 '16 at 14:31
  • 4
    $\begingroup$ Alternatively, Mevik, B.-H. & Wehrens, R. The pls Package: Principal Component and Partial Least Squares Regression in R, Journal of Statistical Software, 18, 1 - 24 (2007) gives a very nice and concise explanation. (which is of course what @amoeba already said, just a bit more readable as it is not restriced to comment format) $\endgroup$ – cbeleites supports Monica May 18 '16 at 14:38
  • 2
    $\begingroup$ @cbeleites: I've re-opened the question then. $\endgroup$ – Scortchi - Reinstate Monica May 21 '16 at 17:29
5
$\begingroup$

I'm mostly using the papers

to extend @amoeba's comment into an answer here:

Let's start with the PLS X

$\mathbf X = \mathbf T \mathbf P' + \mathbf E$ and $\mathbf T = \mathbf X \mathbf W'$

and the Y matrices

$\mathbf Y = \mathbf U \mathbf Q' + \mathbf F$

(outer relations)
(take care to construct the weights $\mathbf W'$ and $\mathbf Q'$ so they refer directly to $\mathbf X$ and $\mathbf Y$, not to deflated matrices!)

Regression can then take place on the X and Y scores: $\hat u = t b$ (inner relation), thus

$\mathbf Y = \mathbf T \mathbf B \mathbf Q' + \mathbf E$

$\mathbf{\hat Y} = \mathbf X \mathbf W' \mathbf B \mathbf Q'$

Now, the last three matrices ($\mathbf W' \mathbf B \mathbf Q'$) are all part of the PLS model parameters. We can therefore introduce one matrix $\mathbf B' = \mathbf W' \mathbf B \mathbf Q'$ which gives PLS coefficients in analogy to the usual MLR coefficients and write
$\mathbf{\hat Y} = \mathbf X \mathbf B'$
which is the usual form of a linear regression model.

Your prediction can either use these "shortcut" coefficients, or the 3 steps of calculating

  1. X scores $\mathbf{\hat T} = \mathbf X \mathbf W'$, then
  2. Y scores $\mathbf{\hat U} = \mathbf{\hat T} \mathbf B$, and finally
  3. $\mathbf{\hat Y} = \mathbf{\hat U} \mathbf Q'$

update: this procedure modeling both $\mathbf X$ and $\mathbf Y$ with latent variables and scores is known as PLS2. In contrast, PLS1 models only one dependent variable $\mathbf y$ (or $\mathbf Y^{(n \times 1)}$ at a time so that no Y-scores are obtained. Multiple dependent variates can be modeled by separate PLS1 models -- one per variate.

Whether multiple PLS1 or a single PLS2 model are better depends on the application, e.g. on whether the dependent variates are correlated and whether an underlying structure with few(er) latent variables is expected.


In practice, you also need to take care of centering (standard practice) and possible scaling (less standard practice) of $\mathbf X$ and $\mathbf Y$.


For cross validation, the prediction works exactly the same way as for unknown cases: you fit the model on your training cases and then predict the left out cases like any other unknown case.

(Assuming this is not asking whether shortcut solutions exist to update a PLS model for exchanging one case during leave-one-out cross validation)

$\endgroup$
  • $\begingroup$ Your explanation makes perfect sense. However, in many of the resources such as in eigenvector.com/evriblog/?p=86 the author is regressing the X scores directly to the Y values rather than Y scores. Is this a matter of my decision? Are there any pros or cons? $\endgroup$ – theGD Oct 3 '16 at 12:07
  • 1
    $\begingroup$ @theGD: the bog entry you link focuses on the $\mathbf X$ side and even though once in a while Barry uses $\mathbf Y$ the formulation uses a a single-column $\mathbf y$ (this is more clearly stated in "The PLS model space revisited" = ref. [4]). A single column doesn't need Y-scores. See also the explanation of the difference between PLS1 (using single column $\mathbf y$ and thus so Y-scores) and PLS2 with Y-scores. With multi-column Y it is your decision whether to build multiple PLS1 models or one PLS2 model. See also e.g. eigenvector.com/faq/index.php?id=93 $\endgroup$ – cbeleites supports Monica Oct 3 '16 at 15:43
  • 1
    $\begingroup$ ... and see the update. $\endgroup$ – cbeleites supports Monica Oct 3 '16 at 17:47
  • $\begingroup$ The first article you refenced is for NIPALS algorithm while the second article uses SIMPLS. In SIMPLS there is no need for Y-Scores in prediction step (both PLS1 and PLS2) and the equation is simply Y = XRQ' where R is the weights(not to be confused with W in NIPALS) and Q is the Y loadings. The confusion that I had long time ago was due to the differences in these algorithms. Furthermore, the particular function "plsregress" in MATLAB library uses SIMPLS algorithm. $\endgroup$ – theGD Feb 3 '17 at 13:05
  • $\begingroup$ Thus, can you please update your answer to indicate that your "shortcut" is available for NIPALS and add a little section for SIMPLS (De Jong, S., 1993. SIMPLS: an alternative approach to partial least squares regression. Chemometrics and Intelligent Laboratory Systems) which is more commonly used in many libraries that I have encountered. IMHO, that update will make things more clear for future readers. Lastly, rather than suggesting an update should I answer my own question? $\endgroup$ – theGD Feb 3 '17 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.