I am trying to implement the closed-form using NumPy and then compare it with sklearn. I can get the same result when there is no fit_intercept (fit_intercept = False). However, when fit_intercept = True, I cannot get the same results even though I have tried several sklearn Ridge solvers. To implement the above formula with NumPy when intercept is not 0, I concatenated 1 to all feature vectors. Below is my code:
#data set (X,y) #using formular N,M = X.shape one = np.ones((N, 1)) Xbar = np.concatenate((one, X), axis = 1) #concatenate 1 to all features vectors I = np.identity(M+1) XT = Xbar.T XTX = XT.dot(Xbar) INV = np.linalg.inv(XTX+alpha*I) beta = INV.dot(XT.dot(y)) print('beta', beta)
and to calculate beta by using sklearn, I implemented the following code:
clf = Ridge(alpha,fit_intercept=True) clf.fit(X, y) print(clf.intercept_, clf.coef_)
However, after trying several solvers of Ridge, I still obtain very different values for weight vectors by the 2 codes. What did I do wrong here?