# Penalized Log-Likelihood - Logistic regression

Equation 5.30 of The Elements of Statistical Learning states that the penalized log-likelihood for a nonparametric logistic regression is:

\begin{align} l(f;\lambda) &= \sum_{i=1}^{N}\big[y_i\log p(x_i) +(1-y_i)\log(1-p(x_i))\big] - \frac12\lambda \int{f''(t)}^2dt \\ &= \sum_{i=1}^{N}\big[y_if(x_i) - \log(1+e^{f(x_i)}\big] - \frac12\lambda \int{f''(t)}^2dt \end{align}

I've read through Negative binomial log-likelihood in penalized regression and What is penalized logistic regression but I haven't come across this formulation yet.

I'm familiar with LASSO and ridge, but the penalized parameter is a vector of $\beta$ coefficients. In this non-parametric scheme, can someone explain why we penalize $\int{f''(t)}^2dt$ instead? I understand that for the non-parametric case, $f(x)$ acts similar to $\beta$ in the parametric case. But it's not obvious to me why this is how it's formulated. Note that

$$f(x) = \log\frac{P(Y = 1|X=x)}{P(Y=0|X=x)}$$

• penalised just means adding a penalty term (which is pretty broad). But the link you refer to refers back to section 5.4 smoothing splines - adding a penalty to enforce smoothness – seanv507 Dec 27 '16 at 20:56
• Right, I understand that it's a penalty term. But why that specific form (integral of square of second derivative)? – ilanman Dec 28 '16 at 13:38
• en.wikipedia.org/wiki/Smoothing_spline – seanv507 Dec 28 '16 at 13:58
• Of course Wikipedia! So, like most things, this penalty is chosen for convenience/out of tradition. Thanks for the link. – ilanman Dec 28 '16 at 14:03