I am performing some simulations of partial least squares. In particular, I have 30 observations split into 20 which are for training and 10 which are for testing. I also have 23 independent variables (all unrelated to each other), the first 18 of which are related to the dependent variable while the last 5 are unrelated to the dependent variable. Then I perform two regressions on the training sample:
- A PLS regression with all 23 independent variables using 18 principal components.
- An OLS regression with only the 18 variables that are related to the dependent variable.
Surprisingly, when I test the two models using the testing sample, the model estimated using 1. consistently produces substantially lower RMSE than the model estimated using 2. It appears that adding variables that are unrelated to the dependent variable improves the model's predictions. The same occurs when I use principal components regression instead of PLS.
What is the cause of this? Is this expected or is it likely due to some mistake in my simulations?
(I also tried estimating 2. using PLS (with 18 components) in order to confirm that the PLS function in R indeed returns the same result as OLS when the number of independent variables is lower than the number of observations.)