# How can I implement lasso in R using optim function

As you know lasso is a popular variable selection method of the form of

$(y-x\beta)'(y-X\beta)+\lambda \sum_i|\beta_i|$

the first is that it is possible to use optim() function in R to minimize this problem? a sample code can be like

x=matrix(rnorm(100),ncol=20)
y=rowSums(x)
f<-function(x,y,l,beta){
beta=as.matrix(beta)
sum((y-x%*% beta)^2) +l*sum(abs(beta))
}
optim(rep(0,ncol(x)),f,method='CG',x=x,y=y,l=1)


Other questions are, 2) is the code above true? 3) how can I force the coefficients to be exactly zero?

PLEASE NOTICE THAT I DONT WANT TO USE PACKAGES LIKE LARS, GLMNET or ... just optim or nlm functions. Thanks

• is this self study? Commented Oct 23, 2014 at 19:00
• By construction, your coefficients will be estimated near 1. To drop the smallest coefficient(s), you will need to experiment with increasing the value of l. But it might be more illustrative to make one of the coefficients smaller than the others so that it is consistently dropped.
– Sycorax
Commented Oct 23, 2014 at 19:28
• @user777 Thank you. why 1? can you explain that more? Commented Oct 23, 2014 at 20:06
• You created y by summing over x.
– Sycorax
Commented Oct 23, 2014 at 20:07
• Sorry, Yes it is. But if generate a sparse matrix and execute the code above you will see that the zero coefficients are not zero. My main challenge is tackling with this coefficients. Maybe I need to change the input data above to sparse data. Commented Oct 23, 2014 at 20:11

With standard algorithms for convex smooth optimization, like CG, gradient descent, etc, you tend to get results that are similar to lasso but the coefficients don't become exactly zero. The function being minimized isn't differentiable at zero so unless you hit zero exactly, you're likely to get all coefficients non-zero (but some very small, depending on your step size). That's why lasso and similar specialized algorithms are useful.

But if you insist on using these algorithms, you can truncate values, e.g., once you've got the "optimal" solution set all betas under 1e-9 or something to zero.

• What about a an algorithm that does not assume smoothness, eg, Nelder-Mead? (Not that I would actually recommend that except in toy examples). Commented Oct 24, 2014 at 6:46
• From my (limited) experience these types of algorithms tend to bounce around a lot, and don't seem to give satisfactory results compared to proximal methods (soft thresholding via gradient or coordinate descent), relative to the amount of effort. But why do it the hard way when there are good methods? Commented Oct 24, 2014 at 7:26