When does partial least squares provide >1 component solutions?

Question

I'm a beginner to using partial least squares analyses, so apologies if this question is a bit basic.

I've been trying out PLS models on my datasets and it usually says that a single component can explain most of the variance in my outcome (when using LOO or CV validation).

However, I've also tried simulating a dataset with 4 principle components with 20 indicators each, and 3 of those components independently predicting the outcome. However, when i run the PLS analyses it usually just says that a single component best explains the outcome variance (when using cross-validation to look at changes in residual error with adding additional components).

My Question Is: I'm confused why a single component solution is working equally well compared to 3 component solution for my simulated data below- as I think i've simulated 3 independent components explaining variance in Y, and a 4th component which is independent of Y.

Doing a principle component regression as expected does find that you need 3-4 components to best explain variance in the outcome, though i know this is not the same as PLS. I've also tried running sparse PLS analyses with mixOmics and found the same result...

N=10000 #Sample Size 

#Simuating 4 factors with fixed component loadings. 
  set.seed(10)
  Factor1_score =rnorm(N)
  Factor1=sapply(runif(20,.7,1),function(x) Factor1_score*.8+rnorm(1000)) #I added the runif function so later i can simulate the effects of varying the degree to which variable loads onto its component- for now its fixed! 
  Factor2_score =rnorm(N)
  Factor2=sapply(runif(20,.7,1),function(x) Factor2_score*.8+rnorm(1000))
  Factor3_score =rnorm(N)
  Factor3=sapply(runif(20,.7,1),function(x) Factor3_score*.8+rnorm(1000))
  Factor4_score =rnorm(N)
  Factor4=sapply(runif(20,.7,1),function(x) Factor4_score*.8+rnorm(1000))
#Bringing variables together 
  TestX=data.frame(scale(cbind.data.frame(Factor1,Factor2,Factor3,Factor4))) #data frame of predictors variables
  colnames(TestX)=c(paste0("F1",1:ncol(Factor1)),paste0("F2",1:ncol(Factor2)),paste0("F3",1:ncol(Factor3)),paste0("F4",1:ncol(Factor4)))
#Simulating Outcome Component - predicted by 3 different factors 
  FactorY=Factor1_score*1+Factor2_score*.3+Factor3_score*.8+rnorm(N,sd=.6)
  Yind=scale(data.frame(sapply(runif(10,.7,1),function(x) FactorY*x+rnorm(N))))
  Yind_Mean=scale(data.frame(apply(Yind,1,mean)))

#Confirming that there are 4 components in the X matrix
  psych::fa.parallel(TestX)

#PLS analysis with pls r package 
pls_test=pls::plsr(as.matrix(I(Yind_Mean)) ~., 10, data = TestX, validation = "CV",scale=TRUE,center=TRUE)
summary(pls_test)
loadings(pls_test)

#PLS analysis with mixOmics r package 
mixOmics_test=mixOmics::pls(cbind.data.frame(TestX),Yind_Mean, ncomp=c(4),scale=TRUE) #keepX=c(6,6))
mixOmics_test_cv=mixOmics::perf(mixOmics_test,validation="Mfold",nepeat=10)

#Principle component regression - as expected a 3-4 component solution works best
pcr_test=pls::pcr(as.matrix(I(Yind_Mean)) ~., 10, data = TestX, validation = "CV",scale=TRUE,center=TRUE)
summary(pcr_test)
loadings(pcr_test)

Answer 1

My Question Is: I'm confused why a single component solution is working equally well compared to 3 component solution for my simulated data below- as I think i've simulated 3 independent components explaining variance in Y, and a 4th component which is independent of Y.

Your simulation has only one data generating process built into $FactorY$:

FactorY=Factor1_score*1+Factor2_score*.3+Factor3_score*.8+rnorm(N,sd=.6)

This means there is only one latent process (typically describe as a latent factor, but this would confuse the use of $Factor$ in your question) for $FactorY$ expected, only one expected combination of the $Factor$#s that correlates with $FactorY$. Adding random noise does not create any additional significant predictors of $FactorY$. Try creating a new $FactorY$

Compare what happens with something like:

FactorY=Factor1_score*1+Factor2_score*.3+Factor3_score*.8+rnorm(N,sd=.6) +
        Factor1_score*0.1+Factor2_score*1+Factor3_score*.3+rnorm(N,sd=.4) +
        Factor1_score*0.3+Factor2_score*.4+Factor3_score*1+rnorm(N,sd=.9)

This will give multiple components that are then weighted and combined to give the final regression coefficients. If you want to play around some more to get deeper understanding try simulating orthogonal or independent data generating processes and see how the components compare to the defined data generating process.

As a reference about multiple underlying contributors to PLS see Henseler which specifically examines the single data generating process (called factor in the paper, so be sure to interpret the meaning of factor differently to your question) vs multiple process scenario for PLS.

Why does PLS behave differently to PCA?

Doing a principle component regression as expected does find that you need 3-4 components to best explain variance in the outcome, though i know this is not the same as PLS

Because PLS reduces the data to balance least squares error in X and Y, while PCA only reduces X. Since the $Factor\#$s are independently created, PCA sees them that way. However, PLS has access to information PCA does not have, $FactorY$ so now PLS can find a connection that PCA could not find. PLS is much more efficient at reducing data to optimise information from in dependent variable than PCA, especially when plenty of noise is present.

For a presentation comparing PCA with PCR and PLS showing how the data reduction differs see here

Explicit answer to the question

When does partial least squares provide >1 component solutions?

PLS provides >1 component solutions when there are more than 1 data generating processes for $FactorY$ that provide a signal greater than the noise.

---

Terminology Note:

Some confusion may be coming from the use of the word 'factor' by in the question.

Based on my original reading of the question the word appeared to be referring to 'independent variable'. In multivariate analysis 'factor' is often reserved for use in reference to uniquely identifiable linear combinations of independent variables such as in factor analysis or when referring to latent factors.

Upon re-reading I can see that maybe the OP's intention was to use factor in the multivariate sense and was simply jumping ahead and directly simulating the factor scores rather than the underlying independent variables. If this is the case then the number of factors included in the PLS is the number of factors given a weighting in the regression coefficients. This is not an acceptable way to compare PLS and PCR as each technique reduces the data differently so PLS factors are not the same as PCA factors. To compare the two you need to go back to simulating the original data.