I'm using Partial Least Squares regression to relate two datasets: X has 40 variables, Y has 54 variables, both have 26 observations. Because I don't have a lot of observations, I'm using a permutation test to determine if the PLS components I get are significant or are overfitting: for each component, I'm comparing the singular values from my PLS to a permutation distribution built by shuffling the rows of X 1000 times (following this paper: https://pubmed.ncbi.nlm.nih.gov/31515054/).

I noticed, though, that the singular values I get in my actual data are significantly lower than those in the permutation distribution (p < 0.05 for the first two components, p < 0.01 for the third one). I've actually gotten a result like this a couple times now relating different data in my dataset with PLS.

How can I interpret a result like this? Does it imply something about the suitability of PLS for my data/something wrong about the procedure? Or (perhaps wishful thinking), might it suggest that there's a nonlinear relationship in the data which reduces the magnitude of the linear relationships found by PLS? Any help would be much appreciated!

More info about the dataset and procedures below:

X and Y had a bit of sparsity to each of them (20-30%), and were standardized before running the model. Most variables in each of them were normal-ish besides this, with a bit of skewness in some of the Y variables. I did a simulation with normally distributed random data of the same size and sparsity, and didn't find anything like the above issue. I'm using plsregress in MATLAB to estimate the components.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.