# Minimizing an elastic net loss function

Reading through Andrew Ng's cs229 notes he shows here on page 10 how you can minimize a loss function $$J\left( \theta \right)$$ representing the sum of least squares by taking the gradient with respect to the weight $$\theta$$ to receive the expression $$X^{T}X\theta - X^{T}\vec{y}$$. My question is, what expression do you receive when minimizing the elastic net loss function $$J\left( \theta \right) = \left\| X\theta-\vec{y}\right\|^2 + \lambda_1\left\| \theta\right\|+ \lambda_2\left\| \theta\right\|^2$$ in a similar fashion?

Looking online, the best that I found is this which near the end gives you a normal equation for $$\theta$$ if your loss function has ridge regression.

• Do you mean a convenient formula for $\hat\beta_{\text{elastic net}}$ like how ordinary least squares has $\hat \beta_{ols}=(X^TX)^{-1}X^Ty?$
– Dave
Sep 10 at 1:59
• Yes @Dave, that is correct Sep 10 at 2:00
• I may have been incorrect about LASSO lacking a closed-form solution. I always thought it lacked one, though, so I’m not sure what I read.
– Dave
Sep 10 at 2:14