3
$\begingroup$

This might be a weird question and I understand that LASSO is mainly using as a variable selection method.

But I want to know that is it possible to check non-linear effects of a LASSO logistic regression model .

In GLM setting , we usually do this by fitting two uni variate models. First model with linear effects and the second model with non linear effects . Then chi square test can be used to check whether the non linear effects are significant.

I tried to do the same thing using glmnet package as follows,

My multiple Lasso logistic regression model :

y <- Smarket$Direction
x <- model.matrix(Direction ~ Lag1 + Lag2 + Lag3 + Lag4 + Volume, Smarket)[, -1]

lasso.mod <- glmnet(x,y, alpha = 1, lambda = 0.002,family='binomial')

lasso.mod$beta
5 x 1 sparse Matrix of class "dgCMatrix"
                 s0
Lag1   -0.065571799
Lag2   -0.035641706
Lag3    0.003564320
Lag4    0.001534812
Volume  0.110035335

Then i tried to check non linear effects for each predictor , by first fitting a uni variate LASSO regression model

y <- Smarket$Direction
x1<- model.matrix(Direction ~ Lag1 , Smarket) 
lasso.mod1 <- glmnet(x1,y, alpha = 1, lambda = 0.002,family='binomial')
lasso.mod1$beta
2 x 1 sparse Matrix of class "dgCMatrix"
                     s0
(Intercept)  .         
Lag1        -0.06312753

After this , is there way to fit univariate lasso model with non linear effects ? Also after fitting that what test can we used to check the non linear effects (can we use chi square test ?) ?

Thank you

Improve this question
$\endgroup$
4
  • $\begingroup$ As far as LASSO is concerned, the idea is mainly to use predictions on a test set / CV to check which model performs better. And if it performs better you can conclude that it probably better describes the data. $\endgroup$
    – user2974951
    Commented Sep 2, 2019 at 13:12
  • $\begingroup$ @user2974951 Thank you for comment. What do you mean by "performs better" ? Does it mean the model with lowest cross validation error ? $\endgroup$
    – student_R123
    Commented Sep 2, 2019 at 13:16
  • 1
    $\begingroup$ The model which maximizes / minimizes your accuracy metric, such as accuracy, misclassification, F1, and so on. Or a scoring rule. $\endgroup$
    – user2974951
    Commented Sep 2, 2019 at 13:19
  • $\begingroup$ what is a univariate LASSO regression? Do you fit the least squares model with LASSO and a single covariate?? $\endgroup$
    – Jonas Striaukas
    Commented Aug 26, 2021 at 19:44

1 Answer 1

Reset to default
1
$\begingroup$

One package you may try is spikeSlabGAM. It's not fast, but it will build nonlinear models and perform selection. To directly answer your question, LASSO is a linear method, but you could make nonlinear terms through transformation (e.g. Lag1^2) and see if they stay in the model.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.