Running some form of regression on an input dataset that exhibits strong multicollinearity can cause unstable regression coefficients, because the regression algorithm can somewhat arbitrarily attribute importance to each predictor in a set of collinear predictors.

Principal Component Regression (PCR) is often used to 'solve' this problem by describing the input dataset according to a set of orthogonal axes - the principal components - which thus do not exhibit multicollinearity.

From what I have read, the PCR procedure is to eliminate the principal components that account for only small variations in the input dataset, then run OLS regression on the principal components. This 'solves' multicollinearity because the coefficients are stable due to the orthogonality of the predictor axes.

However, I have read that doing PCR on all principal components is equivalent to doing OLS on the original data. This suggests that the issue of multicollinearity is therefore not dealt with this 'complete' PCR. I don't understand why? All of the principal components are orthogonal and hence no multicollinearity is present. Indeed, with 'complete' PCR, all of the input data has been included (i.e. no information loss) and therefore using all components should only improve things.

Of course, one of the primary reasons for doing 'incomplete' PCA/PCR is dimensionality reduction, but I want to consider only the issue of multicollinearity. Am I missing something key here?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.