# Bootstrapping standard errors

For research purposes I'm trying to predict the transfer prices of football players. I've implemented several models of which one is Lasso. Now, besides prediction, I would like to measure actual effects of certain variables, for which I've implemented Post-Lasso estimation (Least squares on the non-zero coefficients chosen by Lasso). I've read on several sites, and for one here: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4207812/ that the standard errors are wrong, because they do not take into account the selection procedure which I've used before the least squares estimation. Furthermore, I am dealing with heteroskedasticity and therefore I cannot use the 'normal' standard errors returned by OLS.

Now I was wondering whether or not the bootstrap method: Bagging [1] works to deal with this problem. I would sample with replacements multiple times, run least squares on all these samples and calculate the standard errors over all the coefficients.

My question: Would this work and why (not)?

Thanks a lot!

[1] Breiman, L. (1996). Bagging predictors. Machine learning, 24(2), 123-140.

• You could use a standard non parametric bootstrap. However, I do not see that standard errors are very useful for penalized regression. You are intentionally inducing bias into the model, such that variance is reduced. In order to ontain a variance estimate, you would need to know to the bias. Outside of simmulations, I cant see that this will work Sep 13, 2017 at 7:17
• To reply to @repmat his answer. I'm not trying to estimate the standard errors on my Penalized regression. I'm using the Lasso solely for prediction. I've introduced the Least squares simply to give information regarding the variables that Lasso includes, and what their true values are (for understanding purposes, not prediction). For this reason I need to adjust the standard errors on the least squares estimates, to take into account that I've used a model selection before using the least squares estimation. Regarding your standard non parametric bootstrap. I'm unfamiliar with this technique. Sep 13, 2017 at 7:28
• @IvoHendriks please do not use answers for commenting questions. Answers are meant for answering them. Also please notice that if you have multiple accounts, you can merge them.
– Tim
Sep 13, 2017 at 7:34
• @IvoHendriks that doesnt help. You have biased and inconsistent parameter estimates, inference is essentially worthless in this case. Sep 13, 2017 at 8:08