I am new to glmnet
but would like to apply it to a dataset with binary outcomes. Can you please clarify a few questions for me? Below are the codes and data setup
data(BinomialExample)
set.seed(33)
head(x)
head(y)
cvfit <- cv.glmnet(x, y, type.measure="deviance",alpha=0.5, family="binomial")
(1) In the output below, what is the %Dev” and why is %Dev negatively correlated with $\lambda$?
Specifically, the glmnet vignette said %Dev is the percentage of deviance explained—but on which set? Is it on each holdout set, where %Dev is calculated by running prediction (on the holdout set) with the coefficients generated from training set?
> cvfit$glmnet.fit
Call: glmnet(x = x, y = y, alpha = 0.5, family = "binomial")
Df %Dev Lambda
1 0 0.00000 0.48100
2 1 0.01571 0.43820
3 2 0.03147 0.39930
4 2 0.05559 0.36380
5 2 0.07810 0.33150
6 2 0.09907 0.30210
7 3 0.11930 0.27520
plot(cvfit$lambda,cvfit$glmnet.fit$dev.ratio)
(2) In the plot below, how is the “binomial deviance” (y-axis) corresponding to the optimal $\lambda$ (dash lines) calculated? And how is this “binomial deviance” different from those reported in %Dev as described in question (1)?
I understand that the optimal lambda is chosen to minimize the (cross-validated) deviance. Is the “binomial deviance” shown below defined as averaging over the deviance at fold 1,2…10 (assuming 10-fold CV), where at each fold the deviance is calculated by predicting on the holdout set, using coefficients obtained from training?
plot(cvfit)
(3) On which set is the null deviance (cvfit$glmnet.fit$nulldev
) defined?
It looks like cvfit$glmnet.fit$nulldev
is a constant; is it the null deviance defined on the overall sample?
If glment
has %Dev defined on individual holdout set, wouldn’t it be more convenient to have the null deviance for each holdout set (so that, for example, one can calculate pseudo R-squared at each holdout set), instead reporting null deviance for the overall?
> cvfit$glmnet.fit$nulldev
[1] 137.186