Why l2 norm squared but l1 norm not squared? In the Lasso, and ElasticNet, we use, as penalty, the l1 norm without squaring. But in the ElasticNet and Ridge, we use the l2 norm squared. Why is that, is there a particular reason (computational, optimization dynamics, statistical?)
Because from a purely constrained/penalized convex optimization perspective, looks like both (squared or not squared l1 norm) would solve a similar problem (since $\|x\|_1<D \iff \|x\|_1^2<D^2$), so I guess choosing either l1 squared or plain l1 would only impact the training dynamics (is that correct) ? Could it be then because for non-smooth problems (like with $\ell_1$), one may want to have Lipschitz continuity (which is the case for non-squared l1 but not for squared l1) since we do not have smoothness ?
Curious about this (would be glad to read references).
 A: 
But in the ElasticNet and Ridge, we use the l2 norm squared. Why is that, is there a particular reason (computational, optimization dynamics, statistical?)

A possible reason for the l2 norm being squared in ridge regression (or Tikhonov regularisation) is that it allows an easy expression for the solution of the problem $$\hat\beta = (\textbf{X}^T\textbf{X} + \lambda \textbf{I})^{-1} \textbf{X}^T y$$ where $\textbf{X}$ is the regressor matrix or design matrix, $\lambda$ the scaling parameter for the penalty, $\textbf{I}$ the identity matrix, $y$ the observations, and $\hat{\beta}$ the estimate of the coefficients.

That solution can be derived by taking the derivative of the cost function and setting it equal to zero
$$\nabla_{\hat\beta} \left[  (y - \textbf{X}\hat\beta )^T(y - \textbf{X}\hat\beta ) +  \hat\beta^T  \lambda \textbf{I} \hat\beta \right] = \textbf{X}^T \left (y - \textbf{X}\hat\beta \right)+  \lambda \textbf{I} \hat\beta  = \textbf{0}$$
A: A practical reason for squaring the L2 (that is not specific to ridge regression) is that "squaring" the L2 consists of not bothering to take the square root in the first place.  And since $x^2$ is strictly increasing (for non-negative x), $||f(\textbf{x})||_2$ and $||f(\textbf{x})||_2^2$ will be optimal at the same point, so if the L2 is the optimization target (as opposed to a regularization penalty or something), it's a free speed gain.  Squaring the L1 also doesn't change the optimal point, but it takes more time, so there's no reason to do it.
(If it is being used as a regularization penalty, we might still favor the faster option unless we have a specific reason to want the L2 exactly instead of just something (nonlinearly) proportional to it.)
