6
$\begingroup$

Currently I am learning about variable selection and lasso. I found the paper by Urminsky et al. "Using Double-Lasso Regression for Principled Variable Selection" (2016) which proposes a double lasso variable selection process to identify important IVs and a powerful subset of variables.

It seems to be pretty easy to implement. The following steps are proposed:

  1. Lasso regression of all covariates on DV, to find direct relations between covariates and DV.
  2. Lasso regression of all covariates on IV, to find direct relations between covariates and the focal IV.
  3. Linear regression of all identified important covariates (step 1+2) and focal IV on DV.

Repeat step two to include more focal IVs.

I already asked on cross validated if fitting a normal regression subsequent to a lasso would make sense, and received the answer that this wouldn't be good practice (heres the thread: Lasso for "cherry picking").

What do you think about the double lasso variable selection method?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Intuitively, it should suffer from the same problem as in the linked post. Does the Urminsky paper claim anything about consistency or other interesting properties of the procedure? $\endgroup$
    – Richard Hardy
    May 19, 2016 at 17:32
  • 1
    $\begingroup$ I believe looking at the papers of Belloni et al on LASSO for IV, and on LASSO-OLS should give you many answers. In particular, the authors show that this double selection reduce issues with post-estimation from an imperfect model (in particular, they seem to obtain a uniformly consistent estimator, instead of the pointwise consistence of standard post-selection estimators). Also, the authors suggest to include prior variables, so your repetition of step 2 should be fine. Note other estimators use iterated steps, such as the (multi-step) adaptive lasso, relaxed lasso, thresholded lasso, etc $\endgroup$
    – Matifou
    May 20, 2016 at 0:08

1 Answer 1

Reset to default
3
$\begingroup$

A major advantage of the double selection method is that it is heteroskedasticity robust. Belloni, Chernozhukov and Hansen (ReStud 2014) showed that this is true even if the selection is not perfect.

We propose robust methods for inference about the effect of a treatment variable on a scalar outcome in the presence of very many regressors in a model with possibly non-Gaussian and heteroscedastic disturbances.

[...]

The main attractive feature of our method is that it allows for imperfect selection of the controls and provides confidence intervals that are valid uniformly across a large class of models. In contrast, standard post-model selection estimators fail to provide uniform inference even in simple cases with a small, fixed number of controls. '

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.