12
$\begingroup$

R glm and glmnet use different algorithms.

I notice non trivial differences between the estimated coefficients when I use both.

I am interested in when one is more accurate than another, and the time to solve/accuracy trade off.

Specifically I am referring to the case where one sets lambda=0 in glmnet s.t. it is estimating the same thing as glm.

$\endgroup$
  • 1
    $\begingroup$ You're asking about performance and accuracy differences when lambda=0, where the two should theoretically be identical. I think you should add that into your question. $\endgroup$ – smci Mar 10 '14 at 7:45
14
$\begingroup$

Glmnet is for elastic net regression. This penalises the size of estimated coefficients (via a mix of L1 and L2 penalties). It tries to explain as much variance in the data through the model as possible while keeping the model coefficients small. I found these slides helpful to understand it.

Glm doesn't use a penalty term.

The effect, as I understand it, that with elastic net you may be accepting some bias in return for a reduction in the variance of the estimator. So which is best must depend on how you define 'best' in terms of bias and variance. (E.g. I know glmnet has advantages when you have many features compared to observations)

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ the link appears to be broken $\endgroup$ – ndoogan May 5 '13 at 15:55
  • $\begingroup$ the link works now $\endgroup$ – smci Mar 10 '14 at 7:03
  • 1
    $\begingroup$ Well you'r just explaining what glmnet does - but the OP was referring to the situation when you set lambda=0 in glmnet, in which case the result should in principle return the same as a (nonpenalized) glm (save for some small numerical differences linked with the cyclical coordinate descent fitting method that is used in glmnet). $\endgroup$ – Tom Wenseleers Jan 18 '19 at 0:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.