Why is glmnet ridge regression giving me a different answer than manual calculation? I'm using glmnet to calculate ridge regression estimates. I got some results that made me suspicious in that glmnet is really doing what I think it does. To check this I wrote a simple R script where I compare the result of ridge regression done by solve and the one in glmnet, the difference is significant:
n    <- 1000
p.   <-  100
X.   <- matrix(rnorm(n*p,0,1),n,p)
beta <- rnorm(p,0,1)
Y    <- X%*%beta+rnorm(n,0,0.5)

beta1 <- solve(t(X)%*%X+5*diag(p),t(X)%*%Y)
beta2 <- glmnet(X,Y, alpha=0, lambda=10, intercept=FALSE, standardize=FALSE, 
                family="gaussian")$beta@x
beta1-beta2

The norm of the difference is usually around 20 which cannot be due to numerically different algorithms, I must be doing something wrong. What are the settings I have to set in glmnet in order to obtain the same result as with ridge?
 A: According to https://web.stanford.edu/~hastie/glmnet/glmnet_alpha.html, when the family is gaussian, glmnet() should minimize
$$\frac{1}{2n} \sum_{i=1}^n (y_i-\beta_0-x_i^T\beta)^2
 +\lambda\sum_{j=1}^p(\alpha|\beta_j|
+(1-\alpha)\beta_j^2/2). \tag{1}$$
When using glmnet(x, y, alpha=1) to fit the lasso with the columns in $x$ standardized, the solution for the reported penalty $\lambda$ is the solution for minimizing
$$\frac{1}{2n} \sum_{i=1}^n (y_i-\beta_0-x_i^T\beta)^2
+\lambda \sum_{j=1}^p |\beta_j|.$$
However, at least in glmnet_2.0-13, when using glmnet(x, y, alpha=0) to fit ridge regression, the solution for a reported penalty $\lambda$ is the solution for minimizing
$$\frac{1}{2n} \sum_{i=1}^n (y_i-\beta_0-x_i^T\beta)^2
+\lambda \frac{1}{2s_y} \sum_{j=1}^p \beta_j^2.$$
where $s_y$ is the standard deviation of $y$.  Here, the penalty should have been reported as $\lambda/s_y$.
What might happen is that the function first standardizes $y$ to $y_0$ and then minimizes
$$\frac{1}{2n} \sum_{i=1}^n (y_{0i}-x_i^T\gamma)^2
 +\eta \sum_{j=1}^p(\alpha|\gamma_j|
+(1-\alpha)\gamma_j^2/2), \tag{2}$$
which effectively is to minimize
$$\frac{1}{2n s_y^2} \sum_{i=1}^n (y_i-\beta_0-x_i^T\beta)^2
+\eta \frac{\alpha}{s_y} \sum_{j=1}^p |\beta_j|
+\eta \frac{1-\alpha}{2s_y^2} \sum_{j=1}^p \beta_j^2,$$
or equivalently, to minimize
$$\frac{1}{2n} \sum_{i=1}^n (y_i-\beta_0-x_i^T\beta)^2
+\eta s_y \alpha \sum_{j=1}^p |\beta_j|
+\eta (1-\alpha) \sum_{j=1}^p \beta_j^2/2.$$
For the lasso ($\alpha=1$), scaling $\eta$ back to report the penalty as $\eta s_y$ makes sense.  Then for all $\alpha$, $\eta s_y$ has to be reported as the penalty to maintain continuity of the results across $\alpha$.  This probably is the cause of the problem above.  This is partly due to using (2) to solve (1).  Only when $\alpha=0$ or $\alpha=1$ there is some equivalence between problems (1) and (2) (i.e., a correspondence between the $\lambda$ in (1) and the $\eta$ in (2)).  For any other $\alpha\in(0,1)$, problems (1) and (2) are two different optimization problems, and there is no one-to-one correspondence between the $\lambda$ in (1) and the $\eta$ in (2).
