# Why does this multi-response Guassian LASSO not give a sparse solution?

I tried the glmnet package to learn multi-response Gaussian family. I have looked at the coefficients of the final model. The result is odd. All the features have non-zero coefficients? How is it possible? I used the l1 norm (LASSO).

Before training, I had 20 features in my model, at the end also I have 20 features with non-zero coefficients!

mfit = cv.glmnet(x, y, family="mgaussian", alpha=1)
coef(mfit, s="lambda.min")

$y1 21 x 1 sparse Matrix of class "dgCMatrix" 1 (Intercept) -0.227938007 V1 -0.450036883 V2 0.557660650 V3 0.025960121 V4 -0.006926971 V5 0.938951150 V6 0.028106596 V7 -0.062906327 V8 0.042020881 V9 0.324555833 V10 -1.162720758 V11 1.392068904 V12 0.708822585 V13 0.138470220 V14 -0.361619604 V15 0.263752069 V16 -0.139336945 V17 0.020135397 V18 -0.086938292 V19 0.037916729 V20 0.004525174$y2
21 x 1 sparse Matrix of class "dgCMatrix"
1
(Intercept) -0.152414042
V1           1.816832714
V2          -0.075907117
V3           0.233417492
V4          -0.542780903
V5          -0.038131010
V6          -0.033692294
V7           0.167815325
V8          -0.114406644
V9          -0.202872934
V10          0.023561811
V11          0.192387547
V12         -0.159011058
V13         -0.028944733
V14          0.484381888
V15         -0.009595264
V16         -0.070575757
V17         -0.158257184
V18         -0.636365334
V19         -0.393429761
V20         -0.179587606

$y3 21 x 1 sparse Matrix of class "dgCMatrix" 1 (Intercept) 0.064136138 V1 0.012343756 V2 -0.944652079 V3 -0.055213665 V4 -0.010847049 V5 -0.056957346 V6 0.732330922 V7 -0.016776548 V8 -0.580169864 V9 -0.328945770 V10 1.172705470 V11 0.019684395 V12 0.006571253 V13 0.143321523 V14 -0.017546337 V15 -0.306146331 V16 -0.282589578 V17 1.244944432 V18 0.028064436 V19 0.017680774 V20 -0.243873281$y4
21 x 1 sparse Matrix of class "dgCMatrix"
1
(Intercept)  0.262237827
V1           1.173039843
V2          -0.084647045
V3          -0.070503854
V4           0.630234279
V5           0.021658875
V6          -0.068329527
V7           1.661538220
V8           0.708288249
V9           0.580157907
V10         -0.040516034
V11         -0.251500477
V12         -0.038651852
V13          0.279724140
V14         -0.091477066
V15         -0.557647544
V16         -0.046259431
V17         -1.265200899
V18         -0.008754935
V19          0.205192998
V20         -0.050390759

• There's no rule that says LASSO has to be sparse, only that it can be – shadowtalker Jan 6 '15 at 16:10
• come on, it's not make sense to have all features i your final model with L1 regularization. – user2806363 Jan 6 '15 at 16:56
• that depends entirely on your data – shadowtalker Jan 6 '15 at 16:58
• @user2806363 Think of a simple example: optimize (x - 1)² with L1-regularization lambda = 1. Of course this will not yield x=0 (it will yield x=0.5). So you see, if the L2 error is overpowering the L1 regularization, you will not get zero coefficients. – Owen Mar 29 '17 at 2:02