# Intuition Behind the Elastic Net Penalty

According to Zou et al. 2005, the elastic estimator has the follow formulation.

$$\beta_{elastic} = \arg \min_{\beta} 1/2n\|Y - X\beta\|_2^2 + \alpha \lambda\|z\|_1 + (1-\alpha)\lambda/2 \|z\|_2^2$$

I understand that the elastic net takes the advantage of both the ridge and lasso regression in the penalty term. But I don't understand why the $l_1$-norm and $l_2$-squared norm should interacts linearly via $\alpha$. Does it make more sense to use: $$\beta_{new} = \arg \min_{\beta} 1/2n\|Y - X\beta\|_2^2 + \alpha \lambda\|z\|_1 + (1-\alpha)\lambda \|z\|_2$$

p.s. I understand that adding the coefficient $\frac{1}{2}$ is a equivalent formulation, but I don't understand why we combine a norm and a squared-norm for the elastic net penalty.

For ridge regression, the following two formulations are equivalent via their constrained formulations, or KKT condition:

$$\beta_{ridge1} = \arg \min_{\beta} 1/2n\|Y-X\beta\|_2^2 + \lambda\|\beta\|_2^2$$

$$\beta_{ridge2} = \arg \min_{\beta} 1/2n\|Y-X\beta\|_2^2 + \lambda\|\beta\|_2$$

My question is that why we use $\beta_{ridge1}$ instead of $\beta_{ridge2}$ for the elastic net penalty?

• The penalty term in the second formulation is a sum of the two norm, instead of the sum of a norm and a squared norm. And the links do not fully address my question. – Yang Guo Jul 9 '17 at 20:34
• @gung: $\beta_{ridge1}$ uses squared 2-norm for the penalty term, while $\beta_{ridge2}$ uses 2-norm for the penalty term. – Yang Guo Jul 11 '17 at 23:42
• @Ben: Thank you very much. That link answers my question about the use of the squared-norm and the norm in the elastic net penalty term. Also, I need to point out that p-norm is always convex, and both problem are convex. See: math.stackexchange.com/questions/535825/… – Yang Guo Jul 11 '17 at 23:47
• Ah, of course, my mistake! Thank you! It follows immediately from the triangle inequality that $\beta \mapsto \|\beta\|_2$ is convex. (I was accidentally imagining the function $\beta \mapsto \sqrt{\|\beta\|_2}$ in my comment.) – user795305 Jul 12 '17 at 0:01
• You're right; I see it now. I don't know why I missed it before. – gung - Reinstate Monica Jul 12 '17 at 0:12

This link shows the $\beta_{ridge1}$ and $\beta_{ridge2}$ are equivalent. The only difference for the proof that $\widehat{\beta}_{elastic} = \widehat{\beta}_{new}$to work for the elastic is to change both $\alpha$ and $\lambda$.