# Why is using centered or uncentered data equivalent in ridge regression?

Why is using centered or uncentered data equivalent in ridge regression? In other words, given two ridge regression problems: $$(b',c')=\operatorname*{argmin}_{b,c}\Big[ { \sum_i^{m} (y_i - c - b^Tx_i)^2 + \lambda b^Tb}\Big]$$

$$(b'',c'')=\operatorname*{argmin}_{b,c} \Big[{ \sum_i^{m} (y_i - c - b^T(x_i - \bar{x}))^2 + \lambda b^Tb} \Big]$$ where $\bar{x}$ is the mean of the input data, why does $(b',c')$ correspond to $(b'',c'')$?

I'm writing a piece of code where this thing holds numerically, I was wondering what is the mathematical explanation.

• can we see your code? Nov 27 '15 at 17:21
• Unfortunately the code is for an university assignment, I'm not so happy about posting it on the internet, I'd rather keep it theoretical. If it may help, I have found this sentence: 'If we center the columns of $x$, then the intercept estimate ends up just being $c= \overline{y}$' here stat.cmu.edu/~ryantibs/datamining/lectures/16-modr1-marked.pdf I still don't get it though. Nov 27 '15 at 17:28
• that's correct, the b term just depends on the variance of x. if x is not centered only c changes. remember that the b term is representing the change in y with a unit change in x Nov 27 '15 at 17:28
• @seanv507 thank you. Could you please elaborate a bit? How can you say that b only depends on the variance of x? Nov 27 '15 at 17:32
• sorry, I meant it doesn't depend on the mean. in the univariate case, its the covariance of (x,y)/variance of x, but as I say its easier to think of it as the change in y with one unit change in each component of x (in multivariate case) Nov 27 '15 at 17:51

## 1 Answer

$f(b,c):=\sum_i^m(y_i-c-b^Tx_i)^2+\lambda b^T b$ is equivalent to $g(d,e):=\sum_i^m(y_i-e-d^T (x_i-\bar x))^2+\lambda d^T d$ under the change of variables $d=b,e=c+b^T \bar x$

ie $f(b,c)=g(b,c+b^T\bar x)$.

Therefore they have the same minimisers [same constraints on (b,c) vs (d,e)]. But this change of variables corresponds to using centred or uncentred data.

It should be noted that this only works when the regularisation is not on the constant term. Although regularisation is typically performed as above, some software also penalises the constant/bias term.

• This answer looks very much unfinished; did you intend it to be like that? In its current form it's more of a comment. Would be great if you could extend it to a proper answer. Nov 29 '15 at 22:09
• @amoeba - hope this is better:) Nov 30 '15 at 16:25
• +1. Yes, thanks. I would add that this only works because the intercept $c$ is not penalized. Nov 30 '15 at 16:27