2
$\begingroup$

This is a ridge regression problem. The following two problems are equivalent:

$(w_t, b_\lambda ) = argmin_{w,b}\{\sum_{i=1}^m (y_i-b-w^Tx_i)^2+\lambda w^Tw\} $

$(w_t, b_\lambda )= argmin_{w,b}\{\sum_{i=1}^m (y_i-b-w^T(x_i-\bar x))^2+\lambda w^Tw\} $

where:

  • $\bar x$ is the average of the input data.
  • $\lambda$ defines a trade-off between the error on the data and the norm of the vector $w$ (degree of regularization)
  • I'm assuming $b$ is a bias term

I can't work out why, mathematically, centering the data, has no effect.

Not looking for the answer, just a push in the right direction. Intuitively, I understand that offsetting each data point will have no effect on the final $w$ vector, because the relationships between the data points remains unchanged. Showing this mathematically, however, I'm unsure.

Improve this question
$\endgroup$
3
  • $\begingroup$ Please add the [self-study] tag & read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. $\endgroup$
    – gung - Reinstate Monica
    Commented Jan 17, 2016 at 19:04
  • 2
    $\begingroup$ a 'push in the right direction would be' that one doesn't regularize the constant term :) . I think that answers your question. A fuller discussion can be found in many places. ISLR p 215 towards the bottom is one such place. $\endgroup$
    – meh
    Commented Jan 17, 2016 at 19:06
  • $\begingroup$ I don't think they are the same. If you change the predictors by demeaning them, there's no reason you should get the same weights $w$ $\endgroup$
    – Taylor
    Commented Jan 17, 2016 at 21:02

0

Reset to default