I realise the stupidity of this question, but hear me out.
Imagine a linear regression (e.g. OLS) setting where we perform PCA on all of our independent variables and use all of the resulting principal components (so, number of principal components = number of independent variables) as independent variables instead. From a glance it looks like this will rectify any multicollinearity problem no matter how severe (since all principal components are orthogonal). Moreover, we will not have any problems interpreting the resulting coefficients since they can be projected back into the original coefficient space with no identity loss (since number of principal components we used = number of independent variables) by simply taking a product with principal components. So, a win-win.
Obviously something is wrong with this approach. I just can't get my head around a framework to analyze such situation properly. Any help is appreciated.
To put the question into a perspective, imagine you are tasked with building an OLS model. There are two independent variables which are highly collinear. Usually you would either leave one of them out or perform PCA on them and use the first principal component as a predictor. However, the person who tasked you with building the model is interested in estimating coefficients on both of those variables, so the aforementioned options are not available. Moreover, this person wants the model to meet certain formal criteria, one of which is VIFs below a certain threshold, which is not possible if you include the two highly collinear independent variables as is. Therefore, the only option left is to use PCA on these two variables and use the resulting two principal components as predictors, which allows us to obtain coefficients on both of them in the original coefficient space (by multiplying the coefficients for principal components and principal components themselves) and pass VIF threshold. The person who tasked you with building the model is perfectly fine with this approach, but I can't help but feel that something is off. Are we simply masking the multicollinearity problem here to fool the VIF metric?