# Show that solution to cubic smoothing spline reduces to regular least squares minimization as $\lambda$ approaches infinity

I am asked to show that the solution to a smoothing splines problem of the form $$\text{PRSS}(f,\lambda) = \sum_{i=1}^N\left[y_i-f(x_i)\right]^2 + \lambda \int f''(t)^2 dt,$$ with $$f(x) = \sum_{j=1}^{N+4} \gamma_j B_j(x),$$ where $$B_j(x)$$ is a natural spline, reduces to the regular least-squares fit as $$\lambda \rightarrow \infty$$.

I understand that the first term disappears in the limit and that we are left with minimizing $$\int f''(t)^2 dt$$ but I am not sure how to transform this into something that resembles the least squares. Could anyone please provide me a hint as to how to get started?

• Since PRSS is a differentiable function of the $\gamma_j,$ it is natural to find critical values by equating the gradient of PRSS to zero. – whuber Sep 19 '18 at 17:29

When $$\lambda \rightarrow \infty$$, $$\text{PRSS}(f,\lambda)$$ is finite if and only if $$\int f''(t)^2 dt = 0 \implies f''(t) = 0,$$ which is only true if $$f(t)$$ is a linear function of $$t$$. The objective is then reduced into $$\sum_{i=1}^N\left[y_i-\beta_0 - \beta_1x_i\right]^2,$$ which is the objective for classical linear least-squares regression.