0
$\begingroup$

How do I regress with regularization penalty term lambda * (w - w0)^2 instead of lambda * w^2? Is there any package to do it?

I produced coef w0 with some old data, now I have some new data, I want to regress again and I want the new coef close to w0.

Improve this question
$\endgroup$
2
  • $\begingroup$ The statement "new data" makes me think that you could be interested in bayesian methods, since these methods are explicitly concerned with probability models being updated in the light of new data. $\endgroup$
    – Sycorax ♦
    Commented Sep 5, 2018 at 3:09
  • $\begingroup$ I dont think this is duplicate. The other post is about NN, its accepted answer is just two cost functions, writing down cost function is all the implementation needed. $\endgroup$
    – user3226167
    Commented Sep 6, 2018 at 2:19

1 Answer 1

Reset to default
1
$\begingroup$

Arguably the most straightforward way, from the point of view of using existing software that doesn't give you the option of specifying a non-zero point to shrink the coefficients towards, is to subtract $w0\cdot x$ from the target variable.

Assume a simple regression: $y_i = a + w \cdot x_i + e_i$. If we subtract $w0 \cdot x_i$ from both sides and define $y'_i = y_i - w0\cdot x_i$, we have:

$$y'_i = a + (w-w0)\cdot x_i + e_i $$

Defining $w' = w - w0$ gives us $y'_i = a + w'\cdot x_i + e_i $, and, evidently, $w'$ would be shrunk towards $0$ by the usual sorts of regularization, implying that $w$ would be shrunk towards $w0$.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Suppose my new data have exactly the same distribution as the old ones, the new ceof(i.e. w-w0) should be all zero. I feel very uncomfortable with doing regression with 0 r-squared. $\endgroup$
    – user3226167
    Commented Sep 5, 2018 at 9:30

Not the answer you're looking for? Browse other questions tagged or ask your own question.