# Weight Decay - How to Approach this problem?

How to prove these update rules

Problem

In the augmented error minimization with $\tau = I$ and $\lambda > 0$ , assume that $E_{in}$ is differentiable and use gradient descent to minimize $E_{aug}$ :

$w(t+1) <- w(t) - n\times\delta\times E_{aug}(w(t))$

Problem : I need Show that the update rule above is the same as

$w(t + 1) <- (1 - 2n \lambda) w(t) - n \times \delta \times E_{in}(w(t))$.

Context

The augmented error is regarding regularization :

$E_{aug}(w) = E_{in}(w) + \lambda w^Tw$

where,

$\lambda >= 0$

$\lambda w^Tw$ is the penalty term , the regularizator itself.

In general, the augmented error for a hypothesis $h \in H$ is :

$E_{aug}(h,\lambda,\Omega) = E_{in}(h) + \lambda/N * \Omega (h)$

$\Omega(h)$ is the regularizer which penalizes a particular property of h. And the regularization parameter $\lambda$, the amount of regularization.

• As it stands you're being overly vague and the problem seems to lack a little context. What's "the augmented error minimization"? What do all the symbols mean? What don't you understand at present? What do you already know about this? – Glen_b Jul 5 '16 at 2:44
• @Glen_b i think its better now – KenobiShan Jul 5 '16 at 22:28
• I'll reopen for now, but you still need to give a more specific sense of what help you need. "I want to understand" is not a question, and "how do I approach this" is somewhat too broad for a self-study. Please ask something specific. – Glen_b Jul 6 '16 at 8:19
• I think \$delta \times E \$ should actually be \$delta E \$, meaning differentiate the E with respect to w (rather than multiplying by delta – seanv507 Jul 6 '16 at 8:31