4
$\begingroup$

I understand the approach used for partial least squares for regression (PLS regression) where the PLS components are chosen such that the correlation between the scores of the PLS components of the independent variables and the scores of PLS components of the dependent variables is maximized.

I understand the approach for regression when the dependent variables are continuous. In case the dependent variable is categorical then I learned that the approach is termed partial least squares discriminant analysis (PLS-DA). Is that true?

Is it the same thought process as in PLS regression, except that in PLS-DA the dependent variable would have just two values (for binary classification) and we still go ahead and maximize the covariances across two sets of PLS components?

Improve this question
$\endgroup$
1
  • $\begingroup$ +1. I have edited your post for readability and for consistency of terminology (note that PLS components are not "principal components" -- that term should better be reserved for PCA). Please check that you are happy with all my edits. $\endgroup$ – amoeba Nov 3 '15 at 0:48
1
$\begingroup$

Yes. PLS-DA is basically PLS regression where Y consists of categorical variables. Here is an example of Y matrix with 3 groups each consists of 2 samples (the first row is headers and is not involved in calculations).

Example of Y matrix for PLS-DA

After applying PLS-DA you can obtain a BETA matrix (if you are using SIMPLS algorithm, for example) whose number of columns equals to the number of groups for 2+ groups. There are, however, few differences of PLS-DA from logistic regression and some other classification methods. The predicted Y values might get out of the 0 to 1 range. So you may find values such as 1.1 and -0.3 etc...

Another property of PLS-DA is the sum of the predicted Y values in row is always equal to 1.

Assigning predicted samples to a group can be done in several ways. The most common one is assignation of the sample to the group having the highest value. An alternative approach (Bayesian) is fitting a normal distribution to the predictions of the training set and finding the threshold that minimizes the classification error. The samples, then, can be assigned to the group whose threshold value is exceeded.

Improve this answer
$\endgroup$
12
  • $\begingroup$ What is the standard reference for "PLS-DA"? My favourite guide to various PLS flavours (Rosipal and Kraemer) does not mention this term, so I never know what exactly it is supposed to be. I know Barker and Rayens paper about PLS for discrimination where they suggest to take var(Y) out of the cost function, but they do not use the "PLS-DA" term either. $\endgroup$ – amoeba Jan 26 '17 at 13:20
  • $\begingroup$ I think this is the article: Ståhle, Lars, and Svante Wold. "Partial least squares analysis with cross‐validation for the two‐class problem: A Monte Carlo study." Journal of chemometrics 1, no. 3 (1987): 185-196. $\endgroup$ – theGD Jan 26 '17 at 13:52
  • $\begingroup$ Thanks. But they don't use the "PLS-DA" term... $\endgroup$ – amoeba Jan 26 '17 at 13:54
  • 1
    $\begingroup$ Yes with few possible exceptions. If there are only 2 groups, PLS1 could be suffice. In addition, I have seen some commercial softwares which evaluates x scores rather than y predictions for classification. $\endgroup$ – theGD Jan 26 '17 at 14:32
  • 1
    $\begingroup$ @amoeba: Yes, that's what I understand to be PLS-DA as well: a regression on dummy $y$ that codes the class labels followed by a suitable threshold. However, I find it important to distinguish PLS-DA from PLS-LDA: LDA using PLS X-scores for regularization - which I personally prefer over PLS-DA (see the Barker & Ryans paper mentioned above). The PLS projection being the regression analogue to LDA to means that it is well suited as regularization for LDA, but not necessarily a good replacement. BTW, there's also quite some literature about PLS-LR (logistic regression). $\endgroup$ – cbeleites unhappy with SX Jan 31 '17 at 22:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.