1k views

What is partial least squares (PLS) regression and how is it different from OLS? [duplicate]

Assume we have a simple linear regression model expressed as $$y= X \beta + \epsilon,$$ where $y$ is a vector of size $n \times 1$, $X$ is a matrix of size $n \times p$, $\beta$ is the regression ...
45 views

A mathematical description of Partial Least Squares (PLS)? [duplicate]

I have read many articles about PLS, but I could not understand the mathematical description yet. I know that it is quite similar to principal component regression (PCR), except that it takes into ...
8k views

Why does shrinkage work?

In order to solve problems of model selection, a number of methods (LASSO, ridge regression, etc.) will shrink the coefficients of predictor variables towards zero. I am looking for an intuitive ...
866 views

The limit of “unit-variance” ridge regression estimator when $\lambda\to\infty$

Consider ridge regression with an additional constraint requiring that $\hat{\mathbf y}$ has unit sum of squares (equivalently, unit variance); if needed, one can assume that $\mathbf y$ has unit sum ...
7k views

Principal Component Analysis Eliminate Noise In The Data

Do Principal Component Analysis (PCA) eliminate noise in the data set? If PCA do not eliminate noise in the data set, what actually does PCA do to the data set? Can somebody help me regarding this ...
4k views

Model assumptions of partial least squares (PLS) regression

I am trying to find information regarding the assumptions of PLS regression (single $y$). I am especially interested in a comparison of the assumptions of PLS with regards to those of OLS regression. ...
3k views

Is each of the PCA or PLS components just one of the original variables?

I am confused about what a component is in PCA and PLS. Are the components just the original variables but not necessarily in the same order? For example, in PCA, if I had 8 variables in my data, ...
1k views

Difference between PLS regression and PLS path modeling. Criticism of PLS

This question was asked here but no one gave a good answer. So I think it's a good idea to bring it up again and also I would like to add some more comments/questions. The first question is what is ...
1k views

Objective function of canonical correlation analysis (CCA)

Given two vectors of random variables $X$ and $Y$, Canonical Correlation Analysis (CCA) finds the transformation matrices $A$ and $B$ so that $\operatorname{corr}(A_{1*} X, B_{1*} Y)$ is first maximal,...
350 views

How would Y-aware PCA for binaries look?

I recently stumbled upon Y-aware PCA in the blog of win-vector. They describe how PCA can be adjusted not to explain variation in $X$ but covariation of $X$ and $Y$. This is explained for the case ...