3
$\begingroup$

I have a large amount of vegetation data that has been broken down into 13 habitat classes. I am trying to determine which vegetation tends to fall into or is absent from which habitat with any sort of significance. I have been put onto running a multinomial logistic regression, specifically using glmnet (as I have approximately 200 variables, and only about 260 observations).

Running cv.glmnet using the code:

cv<-cv.glmnet(data,Class,family="multinomial",nfolds=50,standardize=FALSE)

I get a list of numbers that I am struggling to understand, however I found the code:

coef(cv, s=cv$lambda.1se)

Which returns the coefficients for each variable for each habitat class for the lambda that is 1 SE larger than the minimum Lambda value (which as far as I can tell the generally accepted lambda value).

(Intercept)                                              0.7914263664   
Salix                                                    0.0000000000  
Mash                                                     0.0000000000   
Pin                                                      0.0000000000   
Choke                                                    .          
Betula                                                   0.0025260258   
Ideae                                                    0.0000000000   
Leather                                                  0.0000000000

What I'm wondering, using these coefficients, is it possible to state that those values with the largest magnitude (either closest to -1 and +1) are the most important in defining that class, which those close to 0 are unimportant, and those with periods were removed during the cv.glmnet. So in this case the plant "Betula" would be more influential than all others, and "Choke" was so uninfluential that it was removed? Also, no idea what intercept means, but I imagine I can find that one on my own.

$\endgroup$
  • $\begingroup$ The Intercept term is the bias term that was chosen for your model. In the absence of any data for your example, the Intercept is the only value that will be used to predict the class of your example. $\endgroup$ – bourbaki4481472 Sep 4 '15 at 20:47
  • $\begingroup$ Do not combine variable-selection with building the optimal model, especially on the same training-set. You're supposed to keep a holdout set (e.g. 80/20 split). $\endgroup$ – smci Feb 24 '17 at 0:11
  • $\begingroup$ Also, what value of alpha did you use, the default is alpha=1 (Lasso), which sucks. Did you just manually pick an alpha? You're supposed to sweep alpha and lambda to find minimal CV error. $\endgroup$ – smci Feb 24 '17 at 0:12
3
$\begingroup$

First of all, any variable with a coefficient of zero has been dropped from the model, so you can say it was unimportant.

Second of all, you can't really make inferences about the importance of coefficients, unless you scaled them all prior to the regression, such that they all had the same mean and standard deviation (and even then you have to be careful!). If your variables are un-scaled, variables with larger averages will tend to have larger absolute coefficients.

Another option would be to bootstrap sample your data, fit a model to each sample, and calculate confidence intervals around your coefficients.

Finally, how are you choosing the "alpha" parameter for your model?

$\endgroup$
  • $\begingroup$ So let me just check to see if I understand what you are saying. My current request would be potentially dangerous and wouldn't tell me what I needed to know. You suggest instead, doing the same thing but with bootstrapped samples from my data, and determining the confidence intervals around the coefficients. Therefore I.... could interpret the data as I presented in the original question without as much risk depending on the confidence intervals? Sorry if thats way off base. Also, I was using alpha=1 previous to this at the suggestion of someone else (forgot it in my post). $\endgroup$ – HeidelbergSlide Dec 19 '12 at 1:28
  • 3
    $\begingroup$ @Zach: (+1) Trevor Hastie disagrees with the statement you made that "any variable with a coefficient of zero... was unimportant." When I spoke with him this summer, he said that the variable importance measures given by other methods like random forests are much more reliable. If the OP used an elastic net penalty instead of the default of alpha = 1, that can be be somewhat better (see here), but my impression from talking to him was that other methods can be much more reliable. Bootstrapping is, as you said, a good idea here too. $\endgroup$ – David J. Harris Dec 19 '12 at 8:11
  • $\begingroup$ I hate to nag but 1. coefficient of zero could easily be just a variable that is strongly associated with the outcome but also with another, even stronger predictor already in the model - the LASSO tends to pick one good predictor out of a bunch of highly correlated good predictors. 2. I think it should be that variables with higher averages tend to have smaller coefficients. (no?) 3. There are currently no confidence intervals for LASSOs coefficients. Taken from cran.r-project.org/web/packages/penalized/vignettes/… : $\endgroup$ – miura Dec 19 '12 at 8:34
  • $\begingroup$ [...] standard errors can easily be calculated, e.g. using the bootstrap. [...] standard errors are not very meaningful for strongly biased estimates such as arise from penalized estimation methods. Penalized estimation is a procedure that reduces the variance of estimators by introducing substantial bias. The bias of each estimator is therefore a major component of its mean squared error, whereas its variance may contribute only a small part. (section 6) $\endgroup$ – miura Dec 19 '12 at 8:36
  • 1
    $\begingroup$ @David J. Harris: The variable importance metrics provided by random forests actually have the opposite problem: they tend to give MORE weight to correlated predictors! However, you're completely correct about the issue with correlated predictors in a lasso model. However, I would argue that no automated variable selection technique will really be able to tease apart which of 2 correlated predictors is ACTUALLY causing the outcome. At the end of the day, you'd need an experimental study to determine that. =) $\endgroup$ – Zach Dec 19 '12 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.