How to implement the closed form solution of Ridge Regression in Python when intercept is not 0 (fit_intercept=True) without using sklearn?

Question

The well-known closed-form solution of Ridge regression is:

I am trying to implement the closed-form using NumPy and then compare it with sklearn. I can get the same result when there is no fit_intercept (fit_intercept = False). However, when fit_intercept = True, I cannot get the same results even though I have tried several sklearn Ridge solvers. To implement the above formula with NumPy when intercept is not 0, I concatenated 1 to all feature vectors. Below is my code:

#data set (X,y)
#using formular
N,M = X.shape

one = np.ones((N, 1))
Xbar = np.concatenate((one, X), axis = 1)  #concatenate 1 to all features vectors
I = np.identity(M+1)
XT = Xbar.T
XTX = XT.dot(Xbar)
INV = np.linalg.inv(XTX+alpha*I)
beta = INV.dot(XT.dot(y))
print('beta', beta)

and to calculate beta by using sklearn, I implemented the following code:

clf = Ridge(alpha,fit_intercept=True)
clf.fit(X, y)
print(clf.intercept_, clf.coef_)

However, after trying several solvers of Ridge, I still obtain very different values for weight vectors by the 2 codes. What did I do wrong here?

Answer 1

I found the answer here: https://stackoverflow.com/questions/40557569/understanding-ridge-linear-regression-in-sci-kit-learn/40557888#40557888

In summarize, sklearn ridge regression does not add penalty to the intercept term as the analytical formula does.