2
$\begingroup$

I have just started working with the glmnet package in R. I have s a dataset which has about 130,000 features and about 100 rows of data (actually there are about 32,000 rows but I am just using a subset for initial testing). When I print the output of the glmnet model the values for degree of freedom only go from 0 to 1. Can somebody explain why that is?

Here is the code to create the model

myModel = cv.glmnet(data.matrix(modelData), modelData$ACTION,family = "binomial",type.measure = "auc",nfolds = 5)

Any help is appreciated.

Update #1: As pointed out by @user777, the model data should not contain the dependent variable. Here is the new code to create the model.

myModel = cv.glmnet(data.matrix(subset(modelData,select=-ACTION)), modelData$ACTION,family = "binomial",type.measure = "auc",nfolds = 5,alpha = 1)

However now the model just prints the values at 0 and 23. Why is that?

     Df       %Dev    Lambda
  [1,]  0 -5.381e-15 0.0944700
  [2,] 23  1.541e-01 0.0901800
  [3,] 23  2.539e-01 0.0860800
  [4,] 23  3.283e-01 0.0821700
  [5,] 23  3.876e-01 0.0784300
  [6,] 23  4.371e-01 0.0748700
  [7,] 23  4.794e-01 0.0714700
  [8,] 23  5.164e-01 0.0682200
  [9,] 23  5.491e-01 0.0651200
 [10,] 23  5.785e-01 0.0621600
$\endgroup$
2
  • $\begingroup$ I am not sure. As the percentage deviance explained by the model is pretty low I am wondering why it did not try additional features. $\endgroup$
    – Abhi
    Nov 17, 2014 at 3:22
  • $\begingroup$ I think that explains it. Thanks a lot for your help $\endgroup$
    – Abhi
    Nov 17, 2014 at 3:40

1 Answer 1

4
$\begingroup$

Your feature matrix includes your outcome, which is the best predictor of the outcome. You should omit the outcome from your feature set.

myModel <- cv.glmnet(   data.matrix(subset(modelData,select=-ACTION)), 
                    modelData$ACTION,
                    family = "binomial",
                    type.measure = "auc",
                    nfolds = 10)

Deviance may be low, but there's no reason to believe that your features will explain a large proportion of the variance in your outcome.

Package glmnet didn't try additional features because they didn't satisfy the necessary conditions of the fitting algorithm. For more information, you might want to read the supporting articles about the elastic net generally and glmnet specifically.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.