# Why is ridge regression giving different results in Matlab and Python?

Why is the output from Matlab and Python vary for ridge regression? I use the ridge command in Matlab and scikit-learn in Python for ridge regression.

### Matlab

X = [1 1 2 ; 3 4 2 ; 6 5 2 ; 5 5 3];
Y = [1 0 0 1];
k = 10 % which is the ridge parameter

b = ridge(Y,X,k,0)


The coefficients are estimated as

b =    0.3057    -0.0211    -0.0316    0.1741


### Python

import numpy as np
X = np.array([[1, 1, 2] , [3, 4, 2] , [6, 5, 2] , [5, 5, 3]])
Y = np.r_[1,0,0,1].T

from sklearn import linear_model

clf = linear_model.Ridge(alpha=10)
clf.fit(X, Y)

b = np.hstack((clf.intercept_, clf.coef_))


The coefficients are estimated as

 b =  0.716   -0.037   -0.054    0.057


Why is this difference observed?

EDIT: For people who think that centering and scaling is the issue. The input data is not scaled or centered as I had used the scaled parameter as 0 as observed from

b = ridge(Y,X,k,0)


and ridge regression in scikit-learn by default does not do normalization

>>clf
Ridge(alpha=10, copy_X=True, fit_intercept=True, max_iter=None,   normalize=False, solver='auto', tol=0.001)


And here is the Matlab output when it is normalised b = ridge(Y,X,k,1):

 b = -0.0467   -0.0597   0.0870

• Does the discussion at stats.stackexchange.com/questions/23060 answer your question? Dec 10, 2015 at 11:58
• Thanks for the comment. No it does not answer the question. The post tells about centering and scaling of data. In the above problem, in both Matlab and Python, the input data is not scaled and centered. Ideally it should be giving same results. Dec 10, 2015 at 12:05
• Did you notice that the answer to the question I linked tells that Matlab's ridge automatically scales and centers the inputs. If scikit-learn's ridge does not, that explains why the results are different. Dec 10, 2015 at 12:18
• Yes you are right. But in the command 'b = ridge(Y,X,k,0)' I had used the scaled parameter as 0 which does not do the scaling and centering. In the post if you see, the scaled parameter is specified as 1 which does the centering and scaling. And scikit by default does not do scaling and centering as observed from the normalise = false flag, as seen here....Ridge(alpha=10, copy_X=True, fit_intercept=True, max_iter=None, normalize=False, solver='auto', tol=0.001) Dec 10, 2015 at 12:27
• If software is not standardizing all non-constant variables, then it's not doing Ridge Regression--it's doing some ad hoc variation of it. This is an especially important and subtle point whenever interactions are included, because (by the Cauchy-Schwarz inequality) a standardized interaction is never the same as the interaction of standardized variables.
– whuber
Dec 15, 2015 at 20:49

MATLAB always uses the centred and scaled variables for the computations within ridge. It just back-transforms them before returning them. As you have a really small matrix this probably makes a noticeable difference. You can reproduce the Python results in MATLAB easily:

X = [1 1 2 ; 3 4 2 ; 6 5 2 ; 5 5 3];
Y = [1 0 0 1];
k = 10; % which is the ridge parameter
Xn = [ones(4,1), X];

(Xn'*Xn +  diag([0,k,k,k]))\ (Xn'*Y')  %Same as sklearn

ans =
0.7165
-0.0377
-0.0544
0.0572

• I am glad I could help. I would not call it a bug; I am sure they use a particular reference. It be would almost impossible for something like that to go unnoticed by past users and by their internal unit-tests. Probably one of the 1970's reference in the ridge docs does this procedure and in small samples the difference is more pronounced. Dec 10, 2015 at 13:00
• The penalty in ridge regression makes little sense if predictor variables are in different scales. I'd be more inclined to call the scikit-learn default behavior a "bug"; at best, it leaves a serious trap for the unwary.
– EdM
Dec 10, 2015 at 14:17
• @EdM You could say the same for the rest of scikit learn, IMHO.
– Sycorax
Dec 10, 2015 at 15:16
• @apt-getinstallhappyness : absolutely! Ridge regression applies the same penalty to the squares of all coefficients. So unless all predictor variables are in comparable scales, different predictors will be penalized differently and it will matter whether you measure lengths, say, in inches, feet, millimeters, or kilometers. That's why MATLAB normalizes variables to do the regression even though it then back transforms to the original scales.
– EdM
Dec 11, 2015 at 8:30
• @apt-getinstallhappyness : You want to run the ridge regression on the normalized data, then back-transform the coefficients to the original scales. (Don't forget the intercept, too.) Then you can use the back-transformed coefficients for predictions on new data points. As I understand it, that's what the MATLAB procedure does automatically if you provide it data in the original scales to start; check the manual. Absolutely no experience with scikit-learn.
– EdM
Dec 11, 2015 at 18:36