Model assumptions of partial least squares (PLS) regression

Question

I am trying to find information regarding the assumptions of PLS regression (single $y$). I am especially interested in a comparison of the assumptions of PLS with regards to those of OLS regression.

I have read/skimmed through a great deal of literature on the topic of PLS; papers by Wold (Svante and Herman), Abdi, and many others but haven't found a satisfactory source.

Wold et al. (2001) PLS-regression: a basic tool of chemometrics does mention assumptions of PLS, but it only mentions that

1. Xs need not be independent,
2. the system is a function of a few underlying latent variables,
3. the system should exhibit homogeneity throughout the analytical process, and
4. measurement error in $X$ is acceptable.

There is no mention of any requirements of the observed data, or model residuals. Does anyone know of a source that addresses any of this? Considering underlying math is analogous to PCA (with goal of maximizing covariance between $y$ and $X$) is multivariate normality of $(y, X)$ an assumption? Do model residuals need to exhibit homogeneity of variance?

I also believe I read somewhere that the observations need not be independent; what does this mean in terms of repeated measure studies?

Answer 1

When we say that the standard OLS regression has some assumptions, we mean that these assumptions are needed to derive some desirable properties of the OLS estimator such as e.g. that it is the best linear unbiased estimator -- see Gauss-Markov theorem and an excellent answer by @mpiktas in What is a complete list of the usual assumptions for linear regression? No assumptions are needed in order to simply regress $y$ on $X$. Assumptions only appear in the context of optimality statements.

More generally, "assumptions" is something that only a theoretical result (theorem) can have.

Similarly for PLS regression. It is always possible to use PLS regression to regress $y$ on $X$. So when you ask what are the assumptions of PLS regression, what are the optimality statements that you think about? In fact, I am not aware of any. PLS regression is one form of shrinkage regularization, see my answer in Theory behind partial least squares regression for some context and overview. Regularized estimators are biased, so no amount of assumptions will e.g. prove the unbiasedness.

Moreover, the actual outcome of PLS regression depends on how many PLS components are included in the model, which acts as a regularization parameter. Talking about any assumptions only makes sense if the procedure for selecting this parameter is completely specified (and it usually isn't). So I don't think there are any optimality results for PLS at all, which means that PLS regression has no assumptions. I think the same is true for any other penalized regression methods such as principal component regression or ridge regression.

Update: I have expanded this argument in my answer to What are the assumptions of ridge regression and how to test them?

Of course, there can still be rules of thumb that say when PLS regression is likely to be useful and when not. Please see my answer linked above for some discussion; experienced practitioners of PLSR (I am not one of them) could certainly say more to that.

Answer 2

Apparently, PLS does not make "hard" assumptions about the joint distribution of your variables. This means you have to be careful to choose appropriate test statistics (I assume this lack of dependence on variable distributions classifies PLS as a non-parametric technique). Suggestions I found for appropriate statistics are 1) using r-squared for dependent latent variables and 2) resampling methods for assessing stability of estimates.

The main difference between OLS/MLS and PLS is the former typically uses maximum likelihood estimation of population parameters to predict relationships between variables, while PLS estimates values of variables for the true population to predict relationships between groups of variables (by associating groups of predictor/response variables with latent variables).

I'm also interested in handling replicated/repeated experiments, specifically multifactorial ones, however I'm not sure how to approach this using PLS.

Handbook of Partial Least Squares: Concepts, Methods and Applications (page 659, section 28.4)

Wold, H. 2006. Predictor Specification. Encyclopedia of Statistical Sciences. 9.

http://www.rug.nl/staff/t.k.dijkstra/latentvariablesandindices.pdf (pages 4 & 5)

Answer 3

I found a simulation study concerning the influence of non-normality and small sample size in PLS; the authors conclude: "All three techniques [PLS included] were remarkably robust against moderate departures from normality, and equally so."

However, for qualification: "It appears that all three techniques are fairly robust to small to moderate skew or kurtosis (up to skew = 1.1 and kurtosis = 1.6). However, with more extremely skewed data (skew = 1.8 and kurtosis = 3.8), all three techniques suffer a substantial and statistically significant loss of power for both n = 40 and n = 90 (the two sample sizes we tested). For example with n = 90 and medium effect size, regression's power is 76% with normal data, but drops to 53% for extremely skewed data. Under the same conditions PLS's power drops from 75% to 48%, while LISREL drops from 79% to 50%."

(Personally, I would consider those quite modest departures from normality with pretty steep decrements in power.)

Citation: Dale L. Goodhue, William Lewis, and Ron Thompson. Does PLS Have Advantages for Small Sample Size or Non-Normal Data? MIS Quarterly 2012; 36 (3): 891-1001.