2
$\begingroup$

I am working with the famous white wine dataset, and I am trying to fit a logistic model on it, where I also perform ridge regression on this logistic model. Finally, I want to calculate the test error of this model using 10 fold CV. Does this make sense? I ask because trying to do this with R seems to not be easily possible. Here is my code -

# Data pre-processing
winedata <- read.delim("winequality-white.csv", sep = ';')
winedata$quality[winedata$quality< 7] <- "0" #recode
winedata$quality[winedata$quality>=7] <- "1" #recode
winedata$quality <- factor(winedata$quality)# Convert the column to a factor
names(winedata)[names(winedata) == "quality"] <- "good"      #rename 'quality' to 'good'

# I then used 10 fold C to find the best value for lambda for Ridge regression, not adding the code for that here
ridge.model <- glmnet(x, y, alpha = 0, family = "binomial", lambda = bestlam) # fit logistic model with ridge regression. This is the model I want finally
ridge.model.cv.err<- cv.glmnet(x,y, lambda = bestlam, cost1, K=10) # trying to calculate the 10 fold CV of the final model. THIS GIVES ERROR
ridge.model.cv.err$delta # this is what I hoped would give me the test error

This gives the error - Error in cv.glmnet(x, y, lambda = bestlam, cost1, K = 10) : Need more than one value of lambda for cv.glmnet

Am I doing something very wrong here, something that doesn't make sense?

$\endgroup$
5
  • $\begingroup$ what does it mean to perform ridge regression on the logistic model? Is it L2 regularization over the logistic model that you've meant? $\endgroup$
    – gunes
    Apr 4, 2020 at 20:24
  • $\begingroup$ @gunes, I think it would be thought as penalized logistic regression, where I specify the lambda value that specifies the coefficient shrinkage. I calculated this optimum lambda value by doing 10 fold CV for ridge regression on the data $\endgroup$
    – user1995
    Apr 4, 2020 at 20:34
  • $\begingroup$ I read about Penalized Logistic Regression here - sthda.com/english/articles/36-classification-methods-essentials/… $\endgroup$
    – user1995
    Apr 4, 2020 at 20:40
  • $\begingroup$ I think it’s telling you to give all ten $\lambda$ values. What happens if you give it rep(lambda, 10)? $\endgroup$
    – Dave
    Apr 4, 2020 at 21:07
  • $\begingroup$ @Dave number of folds is irrelevant to the number of different lambda values $\endgroup$
    – gunes
    Apr 4, 2020 at 21:24

1 Answer 1

1
$\begingroup$

Finding best $\lambda$ from Ridge Regression and using it in logistic regression is not a good idea. Optimisation objectives significantly differ between the two models. You need to find the best $\lambda$ for the penalised logreg.

The error in your code is actually a good warning. You want to do a cross-validation but there is nothing to cross-validate it says.

$\endgroup$
4
  • $\begingroup$ Thanks for the answer! I understand that finding the best lambda should be done after applying ridge regression to the logistic model, and I will surely do it. But I mainly wanted to ask if cross validation can then be applied to the penalized logistic regression model to get the test error. I am more interested in doing CV for the test error than for finding the best lambda. I have framed the problem from a code point of view here as well - stackoverflow.com/questions/61034771/… $\endgroup$
    – user1995
    Apr 4, 2020 at 21:16
  • 1
    $\begingroup$ Maybe you can set lambda = c(1,1+1e-12) and get your error. Such a small deviance in lambda wouldn't produce a different error. This is hacky, and there are probably better ways to do it (other than coding the cv yourself), but I'm not extremely familiar with glmnet. $\endgroup$
    – gunes
    Apr 4, 2020 at 21:19
  • $\begingroup$ Thanks @gunes. I tried that but the error it gave me using the cvm output of cv.glmnet was much higher than what I have got for this dataset using best subset selection and other methods. I am wondering if cv.glmnet can even be used to get CV test error! Maybe it is only to be used for finding the best lambda parameter $\endgroup$
    – user1995
    Apr 4, 2020 at 21:39
  • $\begingroup$ maybe your best lambda isn‘t the best as I warned you. I don’t think there is any reason for it to give wrong errors. $\endgroup$
    – gunes
    Apr 4, 2020 at 21:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.