2
$\begingroup$

I am having trouble reconciling the different behavior of different Ridge implementations in R. As the following code demonstrates it seems that

  1. MASS:lm.ridge and genridge::ridge return the generative ..beta parameter and not the risk minimizer ..beta.star .
  2. ridge::linearRidge and glmnet::glmnet do return the desired parameters (at least for normalized ..y as required by glmnet::glmnet).

Could the difference be due to the lack of intercept? Maybe the functions are meant for prediction, so that parameter values are meaningless?

# Setup:
BetaStarRidge <- function(beta, lambda) beta/(1+lambda)

..p <- 100L
..n <- 1e4
..x <- rnorm(..p* ..n)  %>% matrix(ncol=..p, nrow=..n)
var(..x) %>% round(,-1)


## y with unit variance (required for glmnet):
..beta <- seq(1:..p)/sqrt(sum((1:..p)^2))/sqrt(2)
..beta^2 %>% sum 
..sigma <- 1e1
..y <- ..x %*% ..beta + rnorm(..n, sd = 1/sqrt(2))
var(..y)  


## Using MASS
library(MASS)
..lambda <- 1e1
..beta.star <- BetaStarRidge(beta = ..beta, lambda = ..lambda)
..ridge.1 <- lm.ridge(..y~..x-1, lambda=..lambda)
..ridge.1$Inter
..coef.1 <- coef(..ridge.1)
plot(..beta~..coef.1);abline(0,1)
plot(..beta.star~..coef.1);abline(0,1)

## Ridge
library(ridge)
..lambda <- 1e1
..ridge.2 <- linearRidge(..y~..x-1, lambda=..lambda)
..coef.2 <- coef(..ridge.2)
..beta.star <- BetaStarRidge(beta = ..beta, lambda = ..lambda)
plot(..beta~..coef.2);abline(0,1)
plot(..beta.star~..coef.2);abline(0,1)


# Using glmnet
library(glmnet)
..lambda <- 1e1
..beta.star <- BetaStarRidge(beta = ..beta, lambda = ..lambda)
..ridge.4 <- glmnet(x =..x, y =..y, family = 'gaussian', alpha = 0, lambda=..lambda, intercept = FALSE)
..coef.4 <- coef(..ridge.4)@x
plot(..beta~..coef.4);abline(0,1)
plot(..beta.star~..coef.4);abline(0,1)

# Using genridge
library(genridge)
..lambda <- 1e1
..beta.star <- BetaStarRidge(beta = ..beta, lambda = ..lambda)
..ridge.5 <- ridge(..y~..x-1, lambda=..lambda)
..coef.5 <- coef(..ridge.5) %>% drop
plot(..beta~..coef.5);abline(0,1)
plot(..beta.star~..coef.5);abline(0,1)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.