I have a set of predictors that clearly suffer from some amount of multicollinearity, so I am using PCA to make the columns of X orthogonal. I am also using this as a way to regularize the subsequent regression by removing components that account for ~0% of the variance.
For example, if I run OLS regression on PCA-transformed data that has 8 predictors, I am then able to use the eigenvectors from the original PCA transformation to get the beta weights for the original 12 predictors. So far, so good.
However, to be able to evaluate the contributions of these predictors to the model fit, I'd like to transform the 95% confidence intervals back into the original space of the 12 predictors. That way, I can use the overall R^2 and associated p-values for the full model to find regressions that are significant, where specific predictors have non-zero contributions.
It is unclear to me how to transform the 95% confidence intervals. If that's not possible, is there another way to evaluate the significance of specific predictors in the original space?