3
$\begingroup$

plots of cost function by number of iterations

Why does L2 regularization smooth the loss surface? went over my head.

$\endgroup$

1 Answer 1

1
$\begingroup$

You can think of minimising the MSE as finding an interpolating surface between the $(x_i, y_i)$ pairs. Regularisation is introduced to reduce the contribution of highly noisy single $ y_i $. From that perspective we'd expect the cost surface to become smoother, as we're biasing it be smoother!

$\endgroup$
3
  • 1
    $\begingroup$ "we're biasing it be smoother"... how? I thought we're biasing weights to be smaller. very confused. $\endgroup$ Feb 19, 2018 at 17:22
  • $\begingroup$ smaller weights lead to a smoother function. How does it lead to a smoother cost function too? $\endgroup$ Feb 19, 2018 at 17:28
  • $\begingroup$ Could you make your question more specific please? $\endgroup$
    – Tom Kealy
    Oct 2, 2021 at 10:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.