How does the LASSO regularization parameter affect the number of features retained?

Question

I am working on a project where I am implementing Lasso regression in R for feature selection and my scenario is as follows.

For the minimum value of $\lambda$ most of the corresponding coefficients are zero (40 out 45 coefficients are zero). It is said that the coefficients will become zero when $\lambda$ is too high. On the contrary for me, the $\lambda$ value is very small (actually to the power of -5) i.e. I have a very small $\lambda$ value and most of the coefficients are zero.

So, I have a few questions listed below:

1. Is this scenario common? Can I take any measures to prevent it?
2. Is selecting by Lasso not suitable for feature selection in my scenario? If so, what are the other methods I can use?

---

Edit

Added R code below:

    coef(cv.glmmod, s = "lambda.min")[which(coef(cv.glmmod, s = "lambda.min") != 0)]    
    > 6.456279e-05 3.838600e-07 1.356334e-05
    colnames(Final_raw)[which(coef(cv.glmmod, s = "lambda.min") != 0)]
    > "smart_1_raw"   "smart_189_raw" "smart_198_raw"
    plot(cv.glmmod) 
    best_lambda <- cv.glmmod$lambda.min
    best_lambda
    > 9.175735e-05
    (plot(cv.glmmod$glmnet.fit)) 

P. S. : I have also tried Ridge and Elastic net and the results were similar as the above.

After Up sampling:

• Output of plot(cv.glmmod)

• Output of plot(cv.glmmod$glmnet.fit)

Answer 1

The penalty parameter $\lambda$ must be chosen after your design matrix is scaled. From section 2.5 of the glmnet paper:

$$N\alpha\lambda_\max = \max_{l \in \{1, \ldots, p\}} \left\vert \sum_{i=1}^n X_{il} y_i \right\vert$$

Note $\alpha = 1$ for lasso regression. $\lambda_\max$ is the value of $\lambda$ for which all coefficients are 0. This function will find $\lambda_\max$ from your input design matrix $X$ and response vector $y$:

find_max_lambda <- function(X, y) {
   mysd <- function(x) sqrt(sum((x-mean(x))^2)/length(x))
   sX <- scale(X, scale = apply(X,2,mysd))
   max(abs(colSums(sX*y/length(y) ) ) )
}