I am going through the LAB section §6.6 on Ridge Regression/Lasso in the book 'An Introduction to Statistical Learning with Applications in R' by James, Witten, Hastie, Tibshirani (2013).
More specifically, I am trying to do apply the scikit-learn
Ridge model to the 'Hitters' dataset from the R package 'ISLR'. I have created the same set of features as shown in the R code. However, I cannot get close to the results from the
glmnet() model. I have selected one L2 tuning parameter to compare. ('alpha' argument in scikit-learn).
regr = Ridge(alpha=11498) regr.fit(X, y)
Note that the argument
glmnet() means that a L2 penalty should be applied (Ridge regression). The documentation warns not to enter a single value for
lambda, but the result is the same as in ISL, where a vector is used.
ridge.mod <- glmnet(x,y,alpha=0,lambda=11498)
What causes the differences?
penalized() from the penalized package in R, the coefficients are the same as with scikit-learn.
ridge.mod2 <- penalized(y,x,lambda2=11498)
Maybe the question could then also be: 'What is the difference between
penalized() when doing Ridge regression?
New python wrapper for actual Fortran code used in R package glmnet