Questions tagged [partial-least-squares]

A class of linear methods for modeling the relationship between two groups of variables, X and Y. Includes PLS regression.

Filter by
Sorted by
Tagged with
33
votes
2answers
5k views

Theory behind partial least squares regression

Can anyone recommend a good exposition of the theory behind partial least squares regression (available online) for someone who understands SVD and PCA? I have looked at many sources online and have ...
15
votes
1answer
8k views

Regression in $p>n$ setting: how to choose regularization method (Lasso, PLS, PCR, ridge)?

I am trying see whether to go for ridge regression, LASSO, principal component regression (PCR), or Partial Least Squares (PLS) in a situation where there are large number of variables / features ($p$)...
16
votes
1answer
3k views

What is the connection between partial least squares, reduced rank regression, and principal component regression?

Are reduced rank regression and principal component regression just special cases of partial least squares? This tutorial (Page 6, "Comparison of Objectives") states that when we do partial least ...
28
votes
1answer
7k views

PCA, LDA, CCA, and PLS

How are PCA, LDA, CCA, and PLS related? They all seem "spectral" and linear algebraic and very well understood (say 50+ years of theory built around them). They are used for very different things (PCA ...
13
votes
2answers
5k views

Model assumptions of partial least squares (PLS) regression

I am trying to find information regarding the assumptions of PLS regression (single $y$). I am especially interested in a comparison of the assumptions of PLS with regards to those of OLS regression. ...
11
votes
1answer
21k views

What is the difference between “loadings” and “correlation loadings” in PCA and PLS?

One common thing to do when doing Principal Component Analysis (PCA) is to plot two loadings against each other to investigate the relationships between the variables. In the paper accompanying the ...
21
votes
2answers
980 views

The limit of “unit-variance” ridge regression estimator when $\lambda\to\infty$

Consider ridge regression with an additional constraint requiring that $\hat{\mathbf y}$ has unit sum of squares (equivalently, unit variance); if needed, one can assume that $\mathbf y$ has unit sum ...
10
votes
1answer
3k views

Why do all the PLS components together explain only a part of the variance of the original data?

I have a dataset consisting of 10 variables. I ran partial least squares (PLS) to predict a single response variable by these 10 variables, extracted 10 PLS components, and then computed the variance ...
7
votes
1answer
358 views

Probabilistic models for partial least squares, reduced rank regression, and canonical correlation analysis?

This question results from the discussion following a previous question: What is the connection between partial least squares, reduced rank regression, and principal component regression? For ...
2
votes
1answer
1k views

What is the relationship between PLS Regression / Discriminant Analysis (PLSR/PLS-DA) and Linear Regression

I'm interested in the problem of feature selection, and I came across PLS-DA, which seems to be a "hack" on PLSR (one more reference). PLSR relates a matrix X to a matrix Y, and PLS-DA relates a ...
10
votes
1answer
1k views

Difference between PLS regression and PLS path modeling. Criticism of PLS

This question was asked here but no one gave a good answer. So I think it's a good idea to bring it up again and also I would like to add some more comments/questions. The first question is what is ...
2
votes
1answer
1k views

Variable selection using cross-validated PLS model when permutation test shows lack of significance

I understand that the permutation test on PLS can help to detect overfitting of the PLS model. Usually if the p-value is greater than a criterion, say 0.05, it means that the model is overfitting and ...
2
votes
1answer
2k views

Are PLS-DA and PLS-LDA the same?

Seems like a trivial question but one for which I can't seem to find an answer. Are PLS-DA (partial least squares discriminant analysis) and PLS-LDA (partial least squares followed by linear ...
5
votes
1answer
8k views

PCA and PLS: testing variables for significance

I'm trying to understand the process for statistical testing for principal component analysis or partial least squares. Step 1. PCA: I feel that I have a not-terrible understanding of PCA: You find ...
2
votes
1answer
208 views

linear path models vs. pls path models (structural equation models)

Assume we have the following linear path model: Structural (inner) model: $Y_{1} = \beta_{1}Y_{2}+\theta_{1}\delta$ Measurement (outer) model: $X_{1} = \lambda_{1}*Y_{1}+\epsilon_{1}\delta$ $X_{2}...
6
votes
1answer
1k views

Mean centering or not in the context of Partial Least Squares

In my current project, I'm using PLS regression on infrared spectra (FTIR). For this I'm using R and the pls function from the plsr package. ...
3
votes
0answers
196 views

PLS “Path Modeling” vs PLS “Regression” [duplicate]

There's a Wiki page for PLS "Path Modeling" and another one for PLS "Regression"... they seem to be saying the same thing, except the first link says PLS is great, and the second says that PLS is ...
3
votes
0answers
2k views

What is partial least squares (PLS) regression and how is it different from OLS? [duplicate]

Assume we have a simple linear regression model expressed as $$y= X \beta + \epsilon,$$ where $y$ is a vector of size $n \times 1$, $X$ is a matrix of size $ n \times p$, $\beta$ is the regression ...
2
votes
1answer
355 views

Partial Least Squares regression - coefficients vs loadings

In partial least squares regression, what is the difference between the regression coefficients and the loadings for each independent variable in each component? Specifically, I understand in evety ...
6
votes
1answer
3k views

Is each of the PCA or PLS components just one of the original variables?

I am confused about what a component is in PCA and PLS. Are the components just the original variables but not necessarily in the same order? For example, in PCA, if I had 8 variables in my data, ...
3
votes
1answer
1k views

Is the PLS-DA approach for categorical variables the same as that used for PLS regression?

I understand the approach used for partial least squares for regression (PLS regression) where the PLS components are chosen such that the correlation between the scores of the PLS components of the ...
3
votes
1answer
1k views

Choosing the number of PLSR components

I am trying to choose how many components to retain in my PLSR. My total variance explained for the response variable is only about 30%, and the first 2 components explain 99% of this. Intuitively I ...
3
votes
1answer
1k views

What's the difference between logistic regression and PLS-DA?

I heard just recently about PLS-DA and I was wondering how it differs from multinomial logistic regression, since logistic regression can be also used for categorical dependent variables.
1
vote
1answer
302 views

How to pick the best model with cross validation?

Based on my understanding the leave one out cross validation is to hold a sample out as the test set and fit a model with remaining data and then calculate the error of prediction of the test sample ...
0
votes
1answer
311 views

categorical predictors in partial least squares

I am interested in running a partial least squares analysis using PROC PLS in SAS 9.4. I understand that, by default, the predictors and response variables in PLS are centered to a mean 0 and scaled ...