41
$\begingroup$

I would like to find predictors for a continuous dependent variable out of a set of 30 independent variables. I am using Lasso regression as implemented in the glmnet package in R. Here is some dummy code:

# generate a dummy dataset with 30 predictors (10 useful & 20 useless) 
y=rnorm(100)
x1=matrix(rnorm(100*20),100,20)
x2=matrix(y+rnorm(100*10),100,10)
x=cbind(x1,x2)

# use crossvalidation to find the best lambda
library(glmnet)
cv <- cv.glmnet(x,y,alpha=1,nfolds=10)
l <- cv$lambda.min
alpha=1

# fit the model
fits <- glmnet( x, y, family="gaussian", alpha=alpha, nlambda=100)
res <- predict(fits, s=l, type="coefficients")
res

My questions is how to interpret the output:

  • Is it correct to say that in the final output all predictors that show a coefficient different from zero are related to the dependent variable?

  • Would that be a sufficient report in the context of a journal publication? Or is it expected to provide test-statistics for the significance of the coefficients? (The context is human genetics)

  • Is it reasonable to calculate p-values or other test-statistic to claim significance? How would that be possible? Is a procedure implemented in R?

  • Would a simple regression plot (data points plotted with a linear fit) for every predictor be a suitable way to visualize this data?

  • Maybe someone can provide some easy examples of published articles showing the use of Lasso in the context of some real data & how to report this in a journal?

Improve this question
$\endgroup$
1
  • $\begingroup$ Why do you run glmnet under the "fit the model" section? Couldn't you use cv for the prediction step as well? $\endgroup$
    – under_the_sea_salad
    Aug 13, 2015 at 18:09

3 Answers 3

Reset to default
22
$\begingroup$

My understanding is that you can't necessarily say much about which variables are "important" or have "real" effects based on whether their coefficients are nonzero. To give an extreme example, if you have two predictors that are perfectly collinear, the lasso will pick one of them essentially at random to get the full weight and the other one will get zero weight.

This paper, which includes one of the authors of glmnet, presents some glmnet-based analyses (see especially: the Introduction, Sections 2.3 and 4.3, and Tables 4 and 5). Glancing through, it looks like they didn't calculate P-valued directly from the glmnet model. They did calculate two different kinds of P-values using other methods, but it doesn't look like they fully trust either of them.

I'm not 100% sure what you're suggesting in terms of plotting methods, but I think it sounds reasonable.

Hope that helps.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Hi David! Thanks for the answer. Would anything speak against using the LASSO for selection of predictors with non-zero coefficients and then use only those predictors in a linear regression model to obtain p-values regarding preditors' significance. E.g. as this paper: ncbi.nlm.nih.gov/pmc/articles/PMC3412288 $\endgroup$
    – jokel
    Aug 23, 2012 at 15:25
  • 4
    $\begingroup$ @jokel I think that what you're suggesting is a special case of the "relaxed lasso", and it can work very well for some purposes. I'm not sure you can trust the p-values you'd get from the procedure you've described, though, since your F statistic or t-statistic won't "know" about the variable selection step you did, and your Type-I error rate will be inflated. One way to think about this: what would the correct number of degrees of freedom be for an F statistic? The total number of variables in the LASSO regression? The number of variables in the secondary regression? Something in between? $\endgroup$
    – David J. Harris
    Aug 23, 2012 at 15:53
  • $\begingroup$ True - so this does not seem to be a valid approach either. Would you have any other idea how to find significant predictors out of 300 independent variables (n>>p like in the above example)? So that in the end I would be able to claim: "predictor X is significantly related to dependent variable Y"? $\endgroup$
    – jokel
    Aug 23, 2012 at 17:38
  • $\begingroup$ My answer to everything where I don't know how to do the calculations is to do randomization. One possibility would be resampling the rows of your data set (e.g. with bootstrapping) and running the LASSO analyses repeatedly. The more often the variable is included in the model, the more likely it is to be important. An even better option might involve samling the rows and columns, which might help avoid "masking" effects. Breiman suggests something in a similar vein in [this pdf] (near Figure 2) (faculty.smu.edu/tfomby/eco5385/lecture/…). $\endgroup$
    – David J. Harris
    Aug 23, 2012 at 18:17
10
$\begingroup$

I just wanted to point out that there is recent work trying to develop a test statistic specifically for the LASSO, which takes into account the feature selection being performed:

A significance test for the lasso. Richard Lockhart, Jonathan Taylor, Ryan J. Tibshirani, Robert Tibshirani. http://arxiv.org/abs/1301.7161

I haven't seen this used in applied work yet however, whereas bootstrapping certainly is used.

Improve this answer
$\endgroup$
0
$\begingroup$

Regarding inference for LASSO or elastic net models have a look at CRAN packages selectiveInference and hdi, they do exactly that whilst taking into account the variable selection step!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.