This is my understanding of glmnet:
if OLS is minimizing RSS, where
$ RSS = \sum(y-\beta x)^2 $
I believe glmnet is minimizing:
$ RSS - \sum(\alpha |\beta_j| + (1-\alpha) \beta_j^2) $ where $\alpha=\lambda_1/(\lambda_1+\lambda_2) $
$\lambda_1$ and $\lambda_2$ come from lasso and ridge regression, but I'm confused if $\lambda_1 = \lambda_2 $ such that
cv.glmnet in glmnet package of R is solving for a single variable (along the whole path) $\lambda$? But then $\alpha = 0.5$ always.
If $\lambda_1 = \lambda_2 $, is the glmnet penalty equivalent to $RSS - \lambda |\beta| - \lambda \beta^2 $
I've read through Hastie et al. 2009 Elements of Statistical Learning and Zou and Hastie 2005 so now I'm trying to get some clarification on the lambdas and alpha. Thanks
I found this to be a useful formulation in Friedman et al (2010) Regularization Paths for Generalized Linear Models via Coordinate Descent.
$$ 1/2N * \sum (y_i - \beta_0 - X\beta)^2 + \lambda P_\alpha (\beta) $$ where $$ P_\alpha (\beta) = \sum (1/2 (1-\alpha) \beta_j^2 + \alpha |\beta_j|) $$ I thought it provided some intuition how lambda and alpha exist together.