Currently I am learning about variable selection and lasso. I found the paper by Urminsky et al. "Using Double-Lasso Regression for Principled Variable Selection" (2016) which proposes a double lasso variable selection process to identify important IVs and a powerful subset of variables.
It seems to be pretty easy to implement. The following steps are proposed:
- Lasso regression of all covariates on DV, to find direct relations between covariates and DV.
- Lasso regression of all covariates on IV, to find direct relations between covariates and the focal IV.
- Linear regression of all identified important covariates (step 1+2) and focal IV on DV.
Repeat step two to include more focal IVs.
I already asked on cross validated if fitting a normal regression subsequent to a lasso would make sense, and received the answer that this wouldn't be good practice (heres the thread: Lasso for "cherry picking").
What do you think about the double lasso variable selection method?