# Neural Nets, Lasso regularization

How does one implement lasso regularization or elastic net on neural networks? (feed forward in particular). I know that closed form solutions for this problems don't exist, still how are they implemented? How do I compute the gradients?

Attention, I'm familiar with both procedures and what they accomplish, what I'm asking is how to implement them, how to formulate the gradients.

This objective function presents one last problem - the L1 norm is not differentiable at 0, and hence poses a problem for gradient-based methods. While the problem can be solved using other non-gradient descent-based methods, we will "smooth out" the L1 norm using an approximation which will allow us to use gradient descent. To "smooth out" the L1 norm, we use $\sqrt{x^2 + \epsilon}$in place of $\left| x \right|$, where ε is a "smoothing parameter" which can also be interpreted as a sort of "sparsity parameter" (to see this, observe that when ε is large compared to x, the x + ε is dominated by ε, and taking the square root yields approximately $\sqrt{\epsilon}$).