# SGD l2 penalty weights update

If you have stochastic gradient descent and you want to update the weights $w_t$ with the L2 norm regularization penalty, how is this math done? I am using sklearn for this and am just wondering how the weight $w_m$ is updated according to $l- \lambda||w||_{2}^{2}$. I read through the sklearn docs but they didn't go into to much detail about how this is actually performed mathematically.

It's the same procedure as SGD with any other loss function. The only difference is that the loss function now has a penalty term added for $\ell_2$ regularization.
The standard SGD iteration for loss function $L(w)$ and step size $\alpha$ is: $$w_{t+1} = w_t - \alpha \nabla_w L(w_t)$$
Say the original loss function was $L_0$, and you add a penalty term for $\ell_2$ regularization. The new loss function is: $$L(w) = L_0(w) + \lambda \|w\|_2^2$$
The gradient is: $$\nabla_w L(w) = \nabla_w \left [ L_0(w) + \lambda \|w\|_2^2 \right ] = \nabla_w L_0(w) + 2 \lambda w$$