0
$\begingroup$

This question already has an answer here:

I just proved to myself why the regularization is added rather than multiplied to loss function.

I did so by taking the MLE formula...

$$\operatorname{argmax}\sum \log(P(x_i\mid\Theta ))$$

and since we know that MAP uses a prior belief distribution...

$$P(\Theta \mid x) = \frac{P(x\mid\Theta )P(\Theta )}{P(x)}$$

We can write MAP as...

$$\operatorname{argmax}\sum log(P(x_i\mid\Theta )P(\Theta)$$

If we redistribute the logs, we can see that $\log(P(\Theta))$ is the regularization terms, as shown below...

$$\log(P(x_i\mid\Theta)) + \log(P(\Theta))$$

but now I would like to show how L2 itself is derived. L2 is defined as...

$$\lambda \sum_k \sum_l W^2_{k,l}$$

which is the element-wise multiplication of the weights. Where did this equation come from? What values for $P(x_i\mid\Theta)$ and $P(\Theta)$, for example, do I need to use to derive this L2 formula? Can someone please explain it to me step-by-step?

$\endgroup$

marked as duplicate by Sycorax, Community Apr 8 at 3:48

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.