I am comparing the perfomance of PCA regression (i.e. regression where original regressors are replaced by few their pr. components) to that of regression with the original regressors in which I added some white noise (mean=0,std=1) to the regressor with the highest correlation coefficient r with the response variable.
I expect a decrease in R squared value when adding noise to one of the independent variables as this increases the unexplained variance in my model.
But I have read about PCA regression as a method able to denoise data as the noise component (or part of them) usually stays in the last components.
So far even if do not use these last components the value of R-squared is more or less the same when not lower.
- Is it really possible to denoise independent variables with PCA-regression?
- If so, how should I proceed?