2
$\begingroup$

I work in a team where everyone uses Stata, and I work in R.

I have created an efficient workflow that allows me to export the results quickly.

The problem I ran into was when implementing the double selection of LASSO. glmnet does not let you work with variables that have missing values. I tried to keep only those variables with complete cases, but doing it with many variables left the data frame without observations. Besides, doing it even with three variables can generate a bias if the missings respond to some particular criterion.

My colleagues use dsregress in Stata which has a special command in that function to ignore the missings (missingok: after fitting lassos, ignore missing values in any othervars not selected, and include these observations in the final model). Is there an alternative in R?

$\endgroup$
4
  • 1
    $\begingroup$ Be a bit more specific. Does 'double' refer to your using elastic net, which requires two penalty parameters? And not that you'll be disappointed in the low probability that lasso or elastic net finding the 'right' model - be willing to do simulations to back up anything you find, as done here. I doubt that you can avoid multiple imputation for your situation, which also gives an opportunity to check feature selection stability. $\endgroup$ Commented Mar 4 at 13:25
  • 1
    $\begingroup$ as a comment to @FrankHarrell -- double selection lasso is an econometric technique in causal inference literature where you do the lasso variable selection for the outcome equation and do the lasso variable selection for the treatment equation and combine the variables selected in hope of some sort of double robustness. The original paper doi.org/10.1093/restud/rdt044 has grown to a cult status with 1800 Google Scholar cites. $\endgroup$
    – StasK
    Commented Apr 24 at 22:39
  • $\begingroup$ That sounds like more of a high likelihood for double non-robustness. Lasso will not find the 'right' variables for either of the two outcomes even if all potential predictors have zero correlation with each other. When there are collinearities things really hit the fan. $\endgroup$ Commented Apr 25 at 12:27
  • $\begingroup$ Well economists have convinced themselves on that one, and it is the discipline expectation that empirical researchers familiarize themselves with the methodology and know how to apply it. I am really just the messenger here. $\endgroup$
    – StasK
    Commented Apr 25 at 22:52

1 Answer 1

3
$\begingroup$

One sort of a weak trick is to replace the missing values with some semi-reasonable number (you can use the mean), and code another variable missing_x = is.na(x) and feed both into your selection. The results will differ between this trick in R and a proper dsregress in Stata, of course.

Why don't you just declare Stata as a knitr engine and call Stata code when you need it inside your R workflow? See https://bookdown.org/yihui/rmarkdown-cookbook/eng-stata.html and links therein.

$\endgroup$
2
  • 1
    $\begingroup$ Using indicator variables for missing will bias estimates and really ruin standard errors. $\endgroup$ Commented Apr 25 at 12:28
  • 1
    $\begingroup$ No doubt @FrankHarrell. That's hardly a great method. That's why I called it weak. $\endgroup$
    – StasK
    Commented Apr 25 at 22:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.