# Difference between LDA and PLS-DA?

Could someone please help by explaining the difference between LDA and PLS-DA? Or are we talking about the same?

First of all, PLS-DA means that you perform a PLS regression and then apply a threshold to assign class labels.

Now, there are two very different situations where this is done:

1. the underlying nature of the problem is metric, and the classes mean that the modeled property is above or below some threshold or limit.
Presence/absence of an analyte (qualitative analysis), legal limits are exceeded or not, ...

2. A "normal" classification problem with cluster-shaped classes is modeled by a dummy regression where the class labels are coded as 0 and 1 or -n1 and +n2.
This is the procedure theGD explains. If I may say so, this is mostly abusing the regression. I assume this is the PLS-DA OP is asking about.

Both are discriminative classification approaches as opposed to one-class classifiers. But other than that the "threshold-type" classification 1. is a very different task from the more usual classification settings in 2.

One important difference is that PLS-DA set up for a situation 1. can deal with classes being "open-ended at the backside" - e.g. analyte concentrations far from the threshold won't disturb the training of the classifier (unless the behaviour of the underlying data over large concentration ranges is more complex).

In contrast, PLS-DA 2. is for cluster-shaped classes (so the same application "group" like LDA).

There is an interesting relationship between LDA and PLS-DA 2.:

PLS-DA using the full PLS model (i.e. all latent variables) produces the same predictions as LDA. OTOH, PLS-DA with only one latent variable produces the same predictions as a Euklidean distance classifier (EDC; i.e. assign the class whose mean is closest).

We can say that PLS-DA regularizes in a way that behaves like "squeezing" the pooled covariance matrix of the LDA into spherical shape the more the fewer latent variables are used.

Keep in mind: there is also PLS-LDA, where LDA is performed in PLS score space rather than the dummy regression of PLS-DA.

Literature:

My personal opinion: if I have cluster-shaped classes and want to go for PLS regularization, I use PLS-LDA rather than PLS-DA.

• Dear @cbeleites-unhappy-with-sx, thanks so much. You've called my attention to a few points that I hadn't thought of. For instance, the relationship with Euclidean distance, which I guess it makes full sense given that you are basically applying a regression model. On your assumption, I was indeed thinking about the classification issue you propose in 2. And thanks for the references, these are quite useful!
– Sos
Commented Mar 30, 2020 at 8:35
• +1 that paper from Brereton is a VERY good reading to understand PLS-DA Commented Mar 30, 2020 at 12:56
• @cbeleites-unhappy-with-sx Can I know that in the context of dimensionality reduction using LDA/FDA. LDA/FDA can start with n dimensions and end with k dimensions, where k < n. Is that correct? Or The output is c-1 where c is the number of classes and the dimensionality of the data is n with n>c.
– aan
Commented May 6, 2020 at 22:20

Both are supervised classification methods.

LDA aims to find projections that aims to minimize within class distance while maximizing between class distance.

PLS-DA is basically PLS regression to class information (I think this kind of class information is called one-hot-encoding of classes in machine learning) and aims to maximize covariance between independent variables and this class information. PLS-DA provides scores, that is also a projection, so LDA-like visualization can be achieved.

• Can I know that in the context of dimensionality reduction using LDA/FDA. LDA/FDA can start with n dimensions and end with k dimensions, where k < n. Is that correct? Or The output is c-1 where c is the number of classes and the dimensionality of the data is n with n>c.
– aan
Commented May 6, 2020 at 22:19