Why is using centered or uncentered data equivalent in ridge regression?

Why is using centered or uncentered data equivalent in ridge regression? In other words, given two ridge regression problems: \begin{equation} (b',c')=\operatorname*{argmin}_{b,c}\Big[ { \sum_i^{m} (y_i - c - b^Tx_i)^2 + \lambda b^Tb}\Big] \end{equation}

$$(b'',c'')=\operatorname*{argmin}_{b,c} \Big[{ \sum_i^{m} (y_i - c - b^T(x_i - \bar{x}))^2 + \lambda b^Tb} \Big]$$ where $\bar{x}$ is the mean of the input data, why does $(b',c')$ correspond to $(b'',c'')$?

I'm writing a piece of code where this thing holds numerically, I was wondering what is the mathematical explanation.

• can we see your code? – TPArrow Nov 27 '15 at 17:21
• Unfortunately the code is for an university assignment, I'm not so happy about posting it on the internet, I'd rather keep it theoretical. If it may help, I have found this sentence: 'If we center the columns of $x$, then the intercept estimate ends up just being $c= \overline{y}$' here stat.cmu.edu/~ryantibs/datamining/lectures/16-modr1-marked.pdf I still don't get it though. – Puzzle Nov 27 '15 at 17:28
• that's correct, the b term just depends on the variance of x. if x is not centered only c changes. remember that the b term is representing the change in y with a unit change in x – seanv507 Nov 27 '15 at 17:28
• @seanv507 thank you. Could you please elaborate a bit? How can you say that b only depends on the variance of x? – Puzzle Nov 27 '15 at 17:32
• sorry, I meant it doesn't depend on the mean. in the univariate case, its the covariance of (x,y)/variance of x, but as I say its easier to think of it as the change in y with one unit change in each component of x (in multivariate case) – seanv507 Nov 27 '15 at 17:51

$f(b,c):=\sum_i^m(y_i-c-b^Tx_i)^2+\lambda b^T b$ is equivalent to $g(d,e):=\sum_i^m(y_i-e-d^T (x_i-\bar x))^2+\lambda d^T d$ under the change of variables $d=b,e=c+b^T \bar x$
ie $f(b,c)=g(b,c+b^T\bar x)$.
• +1. Yes, thanks. I would add that this only works because the intercept $c$ is not penalized. – amoeba Nov 30 '15 at 16:27