Is there any two-stage procedure for elastic net as LASSO? I read this post Why use Lasso estimates over OLS estimates on the Lasso-identified subset of variables?
. It says the LASSO shrinkage causes the estimates of the non-zero coefficients to be biased towards zero. So using OLS after LASSO selection is a recommended method (two-stage LASSO).
Is there any similar rule applying to elastic net? I understand that EN is the combination of ridge + LASSO. Say if I select the number of variables > the number of samples, because some of them are correlated. Then if I apply ridge regression on the selected variables, ridge will also bias the coefficients towards zero, but OLS is not applicable in this case.
So how to resolve this problem?
 A: The adaptive LASSO and the Elastic Net (EN) improve the LASSO in different ways. The adaptive LASSO controls the bias by shrinking larger parameters less, while the EN handles collinearity by incorporating the ridge penalty. The EN can be also extended to shrink parameters by different amounts - the adaptive EN is a combination of the adaptive LASSO and the EN and enjoys the good properties of both methods. Zou & Zhang (2009) suggest fitting the EN in the first stage and using those coefficients to calculate the weights for the adaptive EN.
A: The answer is it depends on what you try to achieve. Post-LASSO OLS estimator is well understood, see e.g. [1]. What has to be said is that you can never be sure that the selected covariates are 'true' covariates - LASSO, like any other estimator (e.g. adaptive LASSO) does mistakes. Oracle properties of adaptive LASSO are derived under very strong assumptions on the covariates - the so-called beta-min condition - which rarely, if at all, happens that the data is such in practice. That being said, the prediction should improve by using post-LASSO OLS or be at least as good as only using LASSO. This is because you reduce the shrinkage bias as you pointed out.
For inference, i.e. stating whether coefficients are significant or not, using post-LASSO OLS is not correct as the results are biased due to omitted variable(s) which LASSO kicks-out mistakenly (as already discussed).
The elastic net is much less understood from the theory point of view, but you may expect similar behavior of this estimator. Therefore, you may try post elastic net OLS and see if it improves your predictions. The same for adaptive versions, although from a theoretical point of view, they are not superior to standard LASSO/elastic net.
Hope this helps!
References:
[1] Belloni, A., & Chernozhukov, V. (2013). Least squares after model selection in high-dimensional sparse models. Bernoulli, 19 (2), 521-547.
https://projecteuclid.org/download/pdfview_1/euclid.bj/1363192037
