5
$\begingroup$

Could someone please help by explaining the difference between LDA and PLS-DA? Or are we talking about the same?

$\endgroup$

2 Answers 2

6
$\begingroup$

First of all, PLS-DA means that you perform a PLS regression and then apply a threshold to assign class labels.

Now, there are two very different situations where this is done:

  1. the underlying nature of the problem is metric, and the classes mean that the modeled property is above or below some threshold or limit.
    Presence/absence of an analyte (qualitative analysis), legal limits are exceeded or not, ...

  2. A "normal" classification problem with cluster-shaped classes is modeled by a dummy regression where the class labels are coded as 0 and 1 or -n1 and +n2.
    This is the procedure theGD explains. If I may say so, this is mostly abusing the regression. I assume this is the PLS-DA OP is asking about.

Both are discriminative classification approaches as opposed to one-class classifiers. But other than that the "threshold-type" classification 1. is a very different task from the more usual classification settings in 2.

One important difference is that PLS-DA set up for a situation 1. can deal with classes being "open-ended at the backside" - e.g. analyte concentrations far from the threshold won't disturb the training of the classifier (unless the behaviour of the underlying data over large concentration ranges is more complex).

In contrast, PLS-DA 2. is for cluster-shaped classes (so the same application "group" like LDA).


There is an interesting relationship between LDA and PLS-DA 2.:

PLS-DA using the full PLS model (i.e. all latent variables) produces the same predictions as LDA. OTOH, PLS-DA with only one latent variable produces the same predictions as a Euklidean distance classifier (EDC; i.e. assign the class whose mean is closest).

We can say that PLS-DA regularizes in a way that behaves like "squeezing" the pooled covariance matrix of the LDA into spherical shape the more the fewer latent variables are used.

Keep in mind: there is also PLS-LDA, where LDA is performed in PLS score space rather than the dummy regression of PLS-DA.

Literature:


My personal opinion: if I have cluster-shaped classes and want to go for PLS regularization, I use PLS-LDA rather than PLS-DA.

$\endgroup$
3
  • 1
    $\begingroup$ Dear @cbeleites-unhappy-with-sx, thanks so much. You've called my attention to a few points that I hadn't thought of. For instance, the relationship with Euclidean distance, which I guess it makes full sense given that you are basically applying a regression model. On your assumption, I was indeed thinking about the classification issue you propose in 2. And thanks for the references, these are quite useful! $\endgroup$
    – Sos
    Commented Mar 30, 2020 at 8:35
  • 2
    $\begingroup$ +1 that paper from Brereton is a VERY good reading to understand PLS-DA $\endgroup$
    – gunakkoc
    Commented Mar 30, 2020 at 12:56
  • $\begingroup$ @cbeleites-unhappy-with-sx Can I know that in the context of dimensionality reduction using LDA/FDA. LDA/FDA can start with n dimensions and end with k dimensions, where k < n. Is that correct? Or The output is c-1 where c is the number of classes and the dimensionality of the data is n with n>c. $\endgroup$
    – aan
    Commented May 6, 2020 at 22:20
4
$\begingroup$

Both are supervised classification methods.

LDA aims to find projections that aims to minimize within class distance while maximizing between class distance.

PLS-DA is basically PLS regression to class information (I think this kind of class information is called one-hot-encoding of classes in machine learning) and aims to maximize covariance between independent variables and this class information. PLS-DA provides scores, that is also a projection, so LDA-like visualization can be achieved.

$\endgroup$
1
  • $\begingroup$ Can I know that in the context of dimensionality reduction using LDA/FDA. LDA/FDA can start with n dimensions and end with k dimensions, where k < n. Is that correct? Or The output is c-1 where c is the number of classes and the dimensionality of the data is n with n>c. $\endgroup$
    – aan
    Commented May 6, 2020 at 22:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.