I am new to
glmnet but would like to apply it to a dataset with binary outcomes. Can you please clarify a few questions for me? Below are the codes and data setup
data(BinomialExample) set.seed(33) head(x) head(y) cvfit <- cv.glmnet(x, y, type.measure="deviance",alpha=0.5, family="binomial")
(1) In the output below, what is the %Dev” and why is %Dev negatively correlated with $\lambda$?
Specifically, the glmnet vignette said %Dev is the percentage of deviance explained—but on which set? Is it on each holdout set, where %Dev is calculated by running prediction (on the holdout set) with the coefficients generated from training set?
> cvfit$glmnet.fit Call: glmnet(x = x, y = y, alpha = 0.5, family = "binomial") Df %Dev Lambda 1 0 0.00000 0.48100 2 1 0.01571 0.43820 3 2 0.03147 0.39930 4 2 0.05559 0.36380 5 2 0.07810 0.33150 6 2 0.09907 0.30210 7 3 0.11930 0.27520 plot(cvfit$lambda,cvfit$glmnet.fit$dev.ratio)
(2) In the plot below, how is the “binomial deviance” (y-axis) corresponding to the optimal $\lambda$ (dash lines) calculated? And how is this “binomial deviance” different from those reported in %Dev as described in question (1)?
I understand that the optimal lambda is chosen to minimize the (cross-validated) deviance. Is the “binomial deviance” shown below defined as averaging over the deviance at fold 1,2…10 (assuming 10-fold CV), where at each fold the deviance is calculated by predicting on the holdout set, using coefficients obtained from training?
(3) On which set is the null deviance (
It looks like
cvfit$glmnet.fit$nulldev is a constant; is it the null deviance defined on the overall sample?
glment has %Dev defined on individual holdout set, wouldn’t it be more convenient to have the null deviance for each holdout set (so that, for example, one can calculate pseudo R-squared at each holdout set), instead reporting null deviance for the overall?
> cvfit$glmnet.fit$nulldev  137.186