0
$\begingroup$

I am using Coxnet package for a dataset of 457 observations and 180 variables and also for another dataset of 457 observations and 25000 variables. 

set.seed(1234)
fit <- Coxnet(x,y,penalty="Lasso")

After creating the model I get some values as follows:

The path of lambda:

  lambda nzero
1  2.144e-01     0
2  1.776e-01     1
3  1.472e-01     1
4  1.220e-01     6
5  1.011e-01     8
6  8.375e-02    10
7  6.940e-02    15
8  5.751e-02    20
9  4.765e-02    24
10 3.949e-02    23

When I use the arguments, nfolds or foldid I get the values as follows:

The path of lambda:

      lambda     cvm    cvse nzero index
1  2.144e-01  -6.091 0.07814     0
2  1.776e-01  -6.068 0.07709     1
3  1.472e-01  -6.051 0.07197     1
4  1.220e-01  -6.042 0.06801     6
5  1.011e-01  -6.029 0.06566     8
6  8.375e-02  -6.019 0.06492    10
7  6.940e-02  -6.015 0.06522    15
8  5.751e-02  -6.012 0.06557    20
9  4.765e-02  -6.004 0.06490    24
10 3.949e-02  -5.998 0.06486    23   max
11 3.272e-02  -6.003 0.06655    27
12 2.711e-02  -6.020 0.06880    31

I read some papers yet couldn't understand what these lambda and nzero stand for.

1) Does these lambda values stand for penalty scores for each variable and does nzero represents the position of the variables?

2) The fit object has a set of Beta values which have zero and non-zero values in it. Are these similar to the coefficient vector in coxph? What does the zero and non-zero values in Beta signify?

3) Can the non-zero coefficient (Beta) be used as a feature selection method to create a coxph model?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ For better general background in LASSO and related modeling approaches, try ISLR Chapter 6, particularly Section 6.2.2 for an explanation of lambda ($\lambda$), and work through Lab 2 in Section 6.6. Although these are presented in the context of linear regression, the basic idea is the same with Coxnet except that partial likelihood rather than least-square error is used as the criterion. Then the explanations of the output in the manual for Coxnet should make sense. $\endgroup$
    – EdM
    Commented Mar 20, 2018 at 17:39
  • $\begingroup$ Thanks. That book was useful. From my understanding, the lasso forces some of the coefficient estimates to be zero when the parameter λ is sufficiently large. Hence, it is much like “variable selection”. Now my question is that, 1) At what point lambda (λ) tends to be very large? I couldn't find this. 2) Can I simply use the non-zero coefficients to extract important variables from my data? 3) Coxnet doesn't predict anything, so after variable selection should I create a coxph model with the selected variables and then perform model assessment? Sorry If I am asking too many questions. $\endgroup$
    – Jesvin Joy
    Commented Mar 21, 2018 at 13:16

1 Answer 1

Reset to default
3
$\begingroup$

  1. In LASSO one generally examines a range of values of $\lambda$ and then chooses a value of $\lambda$ that meets some criterion for an "optimal" model, typically by cross-validation. If you don't specify values for $\lambda$, then programs will typically choose a reasonable range. In your example with Coxnet(), $\lambda$ of 2.144e-01 gave 0 non-zero coefficients (nzero). That's as high in $\lambda$ as you need to go in this case, but the highest useful $\lambda$ value will depend on the particular data being examined. Specifying values for parameters nfolds or foldid tell Coxnet() to perform cross-validation across a range of $\lambda$ values. As appropriate for a measure of optimality in a Cox survival model, Coxnet() then reports "average cross-validation partial likelihood cvm and its standard error cvse, and index with max indicating the largest cvm" (Coxnet manual, page 5). Standard practice would be to choose that value of $\lambda$, 3.949e-02 in your example.

  2. Choose the regression coefficients that are determined at the optimal value of $\lambda$. For example, Coxnet() seems to return coefficients at all tested values of $\lambda$ in fit$Beta and can if requested return the coefficients at the optimal $\lambda$ in fit$Beta0.

  3. Resist the temptation to just take the non-zero predictors from LASSO and set up a brand-new Cox regression model based on them alone. LASSO penalizes the regression coefficients (lowers their absolute magnitudes) to minimize overfitting. If you just take the selected predictors and set up a new model, you have undone this good that LASSO does so your model will end up overfit. Also, the p-values that you get in a new model would not be correct as they would not take into account your prior selection of those predictors based on the data. Just use the coefficients found at the optimal $\lambda$ value. Model assessment might best be done by repeating the entire model-building process on multiple bootstrap samples of the original data, and evaluating how well the multiple models (which in LASSO will likely differ in the particular choices of predictors) work at predictions on the original data set.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Your suggestion was really helpful. As you mentioned I used the coefficient around optimal λ value. In my case, as shown in the example the optimal λ = 3.949e-02 and the corresponding coefficient is -0.02731836. I could get only one coefficient, this coefficient is only for one variable right? also I checked Beta and Bete0 both had almost same values, which one should I use?. My code is as follows: optimal.lambda <- fit$lambda.opt lambda.index <- which(fit.obj$lambda==optimal.lambda) optimal.beta <- fit$Beta[lambda.index]. Is this right or am I missing something? $\endgroup$
    – Jesvin Joy
    Commented Mar 22, 2018 at 16:37
  • $\begingroup$ @JesvinJoy for each lambda you should have a set of coefficients for ALL the predictors, with many set to 0 via LASSO. If you are only getting one coefficient for a value of lambda then there is something wrong. I don't use Coxnet (I use glmnet, which provides the same basic structure for many types of models), so I'm not sure how indexing of Coxnet output is done for fit$Beta. It seems that fit$Beta0 on its own should provide a list or vector of all coefficients at the optimum lambda. $\endgroup$
    – EdM
    Commented Mar 22, 2018 at 16:52
  • $\begingroup$ My first dataset has 180 predictors and I got exactly 180 coeff each for one predictor, with many set to 0. Another dataset had 1400 genes and the results were same. I checked fit$Beta and fit$Beta0 both had same values of nonzero and zero coeff at the same location. The 180 predictors dataset had 23 nonzero coefficients in both fit$Beta and fit$Beta0 making it seem like duplicates. The 1400 predictors dataset had slight difference, one with 23 nonzero and fit$Beta0 with only 20 nonzero coeff. However, the 20 fit$Beta0 coeff values were exactly similar to the one in fit$Beta. $\endgroup$
    – Jesvin Joy
    Commented Mar 23, 2018 at 16:58
  • $\begingroup$ @JesvinJoy the number of non-zero coefficients pretty much makes sense. If half of your 457 cases had events, then 20 non-zero coefficients would be on the order of 10 events per predictor, which should prevent overfitting. I don't know why fit$Beta0 would return a slightly different set of coefficients than fit$Beta evaluated at the same value of lambda; I don't use Coxnet myself. You might want to check the source code in R. If you need to explain your modeling to someone else, it might be easiest just to use fit$Beta0. That would also be the easiest to automate for bootstrap validation. $\endgroup$
    – EdM
    Commented Mar 23, 2018 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.