14
$\begingroup$

R glm and glmnet use different algorithms.

I notice non trivial differences between the estimated coefficients when I use both.

I am interested in when one is more accurate than another, and the time to solve/accuracy trade off.

Specifically I am referring to the case where one sets lambda=0 in glmnet s.t. it is estimating the same thing as glm.

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ You're asking about performance and accuracy differences when lambda=0, where the two should theoretically be identical. I think you should add that into your question. $\endgroup$
    – smci
    Commented Mar 10, 2014 at 7:45

1 Answer 1

Reset to default
13
$\begingroup$

Glmnet is for elastic net regression. This penalises the size of estimated coefficients (via a mix of L1 and L2 penalties). It tries to explain as much variance in the data through the model as possible while keeping the model coefficients small. I found these slides helpful to understand it.

Glm doesn't use a penalty term.

The effect, as I understand it, that with elastic net you may be accepting some bias in return for a reduction in the variance of the estimator. So which is best must depend on how you define 'best' in terms of bias and variance. (E.g. I know glmnet has advantages when you have many features compared to observations)

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Well you'r just explaining what glmnet does - but the OP was referring to the situation when you set lambda=0 in glmnet, in which case the result should in principle return the same as a (nonpenalized) glm (save for some small numerical differences linked with the cyclical coordinate descent fitting method that is used in glmnet). $\endgroup$
    – Tom Wenseleers
    Commented Jan 18, 2019 at 0:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.