13
$\begingroup$

I am trying to find information regarding the assumptions of PLS regression (single $y$). I am especially interested in a comparison of the assumptions of PLS with regards to those of OLS regression.

I have read/skimmed through a great deal of literature on the topic of PLS; papers by Wold (Svante and Herman), Abdi, and many others but haven't found a satisfactory source.

Wold et al. (2001) PLS-regression: a basic tool of chemometrics does mention assumptions of PLS, but it only mentions that

  1. Xs need not be independent,
  2. the system is a function of a few underlying latent variables,
  3. the system should exhibit homogeneity throughout the analytical process, and
  4. measurement error in $X$ is acceptable.

There is no mention of any requirements of the observed data, or model residuals. Does anyone know of a source that addresses any of this? Considering underlying math is analogous to PCA (with goal of maximizing covariance between $y$ and $X$) is multivariate normality of $(y, X)$ an assumption? Do model residuals need to exhibit homogeneity of variance?

I also believe I read somewhere that the observations need not be independent; what does this mean in terms of repeated measure studies?

$\endgroup$
  • $\begingroup$ The link to Wold. et al is incorrect. Is this the one it should be? libpls.net/publication/PLS_basic_2001.pdf $\endgroup$ – emudrak Mar 2 '15 at 16:45
  • $\begingroup$ A client had a reviewer comment to a paper that said something line "show you checked the linearity assumption." How would you do this? $\endgroup$ – emudrak Mar 2 '15 at 16:52
5
$\begingroup$

When we say that the standard OLS regression has some assumptions, we mean that these assumptions are needed to derive some desirable properties of the OLS estimator such as e.g. that it is the best linear unbiased estimator -- see Gauss-Markov theorem and an excellent answer by @mpiktas in What is a complete list of the usual assumptions for linear regression? No assumptions are needed in order to simply regress $y$ on $X$. Assumptions only appear in the context of optimality statements.

More generally, "assumptions" is something that only a theoretical result (theorem) can have.

Similarly for PLS regression. It is always possible to use PLS regression to regress $y$ on $X$. So when you ask what are the assumptions of PLS regression, what are the optimality statements that you think about? In fact, I am not aware of any. PLS regression is one form of shrinkage regularization, see my answer in Theory behind partial least squares regression for some context and overview. Regularized estimators are biased, so no amount of assumptions will e.g. prove the unbiasedness.

Moreover, the actual outcome of PLS regression depends on how many PLS components are included in the model, which acts as a regularization parameter. Talking about any assumptions only makes sense if the procedure for selecting this parameter is completely specified (and it usually isn't). So I don't think there are any optimality results for PLS at all, which means that PLS regression has no assumptions. I think the same is true for any other penalized regression methods such as principal component regression or ridge regression.

Update: I have expanded this argument in my answer to What are the assumptions of ridge regression and how to test them?

Of course, there can still be rules of thumb that say when PLS regression is likely to be useful and when not. Please see my answer linked above for some discussion; experienced practitioners of PLSR (I am not one of them) could certainly say more to that.

$\endgroup$
  • $\begingroup$ What about normality and the independence of sampling? $\endgroup$ – WCMC Jan 25 '16 at 0:10
3
$\begingroup$

Apparently, PLS does not make "hard" assumptions about the joint distribution of your variables. This means you have to be careful to choose appropriate test statistics (I assume this lack of dependence on variable distributions classifies PLS as a non-parametric technique). Suggestions I found for appropriate statistics are 1) using r-squared for dependent latent variables and 2) resampling methods for assessing stability of estimates.

The main difference between OLS/MLS and PLS is the former typically uses maximum likelihood estimation of population parameters to predict relationships between variables, while PLS estimates values of variables for the true population to predict relationships between groups of variables (by associating groups of predictor/response variables with latent variables).

I'm also interested in handling replicated/repeated experiments, specifically multifactorial ones, however I'm not sure how to approach this using PLS.

Handbook of Partial Least Squares: Concepts, Methods and Applications (page 659, section 28.4)

Wold, H. 2006. Predictor Specification. Encyclopedia of Statistical Sciences. 9.

http://www.rug.nl/staff/t.k.dijkstra/latentvariablesandindices.pdf (pages 4 & 5)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.