Is it okay to residualize a variable out of my dependent variable, to deal with multicollinearity? Here is my situation. I have n predictors of interest, and two control variables.
If I put them all together in a multiple regression, I get issues with colinearity (i.e., VIFs are very high, and the coefficients don't make sense).
It seems that my control variables are causing the colinearity. If I run colineaity diagnositics on just my predictors of interest they seem fine. But when I run the diagnosticts on the predictors of interest AND the control variables, I get high VIF values.
I tried predicting my dependent variable with just my control variables and saving the residuals (in other words residualizing my dependent variable). If I then predict these with my predictors of interest, the results are very interpretable, and I don't have any colineaity issues.
Is this an acceptable way to deal with this?
I'm talking specifically about multiple regression and stepwise regression.
Now I am also conducting a LASSO, to compare results. Is there any harm in also doing the same there?
Long story short. I use my control variables to residualize my dependent variable. I then run a model predicting the residualized dependent variable with my variables of interest. Is that okay?
 A: Fitting the model on the residuals from your control variables is the same as fitting the model all together after orthogonalizing the study variables with respect to controls (it's the same model, but partly rotated). Not the best thing by interpretability standards, but absolutely legit all in all.
Compared to standard OLS, you have two pieces of the same model, the second of which has been rotated to be orthogonal to the first. The covariance structure of $\beta$ will be rotated as well, with the effect of reducing the s. errors as well as the covariances. On the other hand, the estimated effects of the second piece of the model are not the same thing, they are now effects conditioned on the first part of the model (you fit them on the first part residuals, don't you?).
You can also use LASSO, I don't see any reason not to.
edit: I mean apply LASSO on the second part of the model, after running the first one and taking the residuals, that is fine. If you run LASSO on the whole model all together, collinearity is an issue: LASSO tends to select the most effective predictor and bring the other collinear ones to 0, which is generally not ideal. I'll restate it: if you want to do variable selection on the study variables using the residuals of the control variables model, you can do it with LASSO. In any case (LASSO or not) the collinear study variables, to be selected, must show a significant predictive effect in their component that is orthogonal to the control variables. Estimating the model all together, with or without shrinkage, will give a much different result.
