I ran three regularization methods, lasso, ridge, and elastic net. Lasso was able to get the best accuracy, so I'm selecting it. Is there a way to calculate odds ratio from the coefficients? Does it make sense to do it in glmnet?
I took the following steps:
train.control <- trainControl(method = "repeatedcv", number = 10, repeats = 5, allowParallel = T, verboseIter = T) set.seed(1234) lasso_model <- train(traget~ ., trainTransformed[,-2], trControl = train.control, method = "glmnet", tuneGrid = expand.grid(alpha = 1, lambda = seq(0.0001, 0.05, length = 5)), family = "binomial")
Plot and predict the model
plot(lasso_model$finalModel, xvar = "lambda", label = T) plot(lasso_model$finalModel, xvar = "dev", label = T) plot(varImp(lasso_model, scale = F)) p.lasso.pred <- predict(lasso_model, testTransformed) p.lasso.pred.cm <- confusionMatrix(p.lasso.pred, testTransformed$BMK_R_Derailment, mode = "prec_recall")
Now, all tutorials that I've read stops at this point. I'm really confused as to whether to stop here, or take the features from lasso with coefficients > 0 and run logistic again to get the odds ratio for the coefficients.
And I also did that. However, most of the variables are not significant (which is fine). Then should I select the variables that are significant and do the regular (step-wise - not sure if I should do this) logistic regression? or leave the model as is because lasso produced those features?