2
$\begingroup$

When learning about Regularization in Regression, I am used to seeing this in the following form (https://en.wikipedia.org/wiki/Regularization_(mathematics)):

enter image description here

When watching this video (https://www.youtube.com/watch?v=0nYiGXooG4I), I see a similar equation as provided above:

enter image description here

The thing which I don't understand is that when comparing the second equation to the first equation - why is there an integral in the second equation and why is there a second derivative of the function?

Thanks!

$\endgroup$
1
  • $\begingroup$ $R(f)$ is a general penalty, and $R(f) = \int f''(x)^2dx$ is a specific penalty. $R(f)$ could be something else, like $R(f) = \int f'(x)dx$, $R(f) = \sup_{x}|f(x)|$, or $R(f) = ||f||_{\mathrm{RKHS}}$, where $||.||_{\mathrm{RKHS}}$ measures how quickly $f$ is oscillating. $\endgroup$ Commented Oct 15, 2022 at 2:44

1 Answer 1

1
$\begingroup$

This video talks about fitting smoothing splines, so they want the splines in $f$ to be smooth. Since $f^{\prime\prime}$ is measuring the curvature of $f$, $\int (f^{\prime\prime}(x))^2dx$ is (proportional to) the average squared curvature.

So, this optimization penalizes large average curvature; apparently, smooth here means to be rather flat.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.