0
$\begingroup$

I am comparing the perfomance of PCA regression (i.e. regression where original regressors are replaced by few their pr. components) to that of regression with the original regressors in which I added some white noise (mean=0,std=1) to the regressor with the highest correlation coefficient r with the response variable.

I expect a decrease in R squared value when adding noise to one of the independent variables as this increases the unexplained variance in my model.

But I have read about PCA regression as a method able to denoise data as the noise component (or part of them) usually stays in the last components.

So far even if do not use these last components the value of R-squared is more or less the same when not lower.

My question:

  • Is it really possible to denoise independent variables with PCA-regression?
  • If so, how should I proceed?
Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ "PCA regression" can eliminate that noise indeed if you conceptualize (independently of the existence of Y variable) your p observed independent variables as few m constructs plus p-m dimensional noise (error) orthogonal to those m ones. In your exercise of adding noise you added it right to the strongest predictor of Y; that action is incomparable in theory and consequences with "PCA regression". $\endgroup$
    – ttnphns
    Commented Sep 22, 2017 at 10:31
  • $\begingroup$ Not sure to have understood your answer. "you added it right to the strongest predictor of Y; that action is incomparable in theory and consequences with PCA regression" Should I add noise to all the predictors and then take the components with better correlation with Y and highest eigenvalues? @ ttnphns $\endgroup$
    – gis20
    Commented Sep 22, 2017 at 14:23

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.