# How to show that smoothing spline fit preserves the local regression part of the fit

We need to show that a smoothing spline of $$y_i$$ to $$x_i$$ retains the local regression part of the fit.
For linear regression, this problem seems trivial because it is relatively easy to move from $$y_i = \hat y_i + r_i$$ to $$S y_i = \hat y + Sr$$ where $$S$$ is smoothing matrix. But I don't really understand how to do the same for the case when $$\hat y_i$$ is no linear but local regression.
Can you give a hint?

• Can you explain what you mean by 'retaining the local regression part'? And can you indicate what y_hat and r are? – Sebastiaan Mar 8 at 11:53
• @Sebastiaan Unfortunately, the task itself does not clarify this moment. However, my understanding is the following: $\hat y$ is the prediction of local regression fitted on the data with response $y$. $r$ is kind of error or difference between $y$ and $\hat y$. – Bruh Mar 10 at 15:44
• This is exercise 9.1 and the of the chapter - stat.auckland.ac.nz/~yee/784/files/ch09AdditiveModelsTrees.pdf – Bruh Mar 10 at 15:51
• Sorry but I don't feel like helping you with your homework if you cannot even be bothered to make an effort explaining the problem. – Sebastiaan Mar 10 at 17:57
• @Sebastiaan I have already provided my understanding of $\hat y$ and $r$ in the comment above. Also, I provided the link to the exercise in order to show that the task doesn't explain anything more. – Bruh Mar 11 at 6:10

## 2 Answers

Your idea of multiplying the formula by $$S$$ from both sides makes sense. And the result for the local regression will be the same. However, the form of the smoothing matrix will be different. You might know that for linear regression $$S$$ is the following:

$$S = X(X^TX)^{-1}X^T$$ Think what is $$S$$ for local regression.

Then the only thing that you need to do is to show that $$S$$ for local regression is idempotent.

I am assuming that your smoothing spline $$f_s$$ is

$$f_s = \arg\min_f \sum_{i=1}^n {(y_i-f(x_i))}^2 + \lambda \int {(f''(x))}^2 \mathrm{d} x,\qquad\text{(1)}$$

and that by local regression you mean $$\hat y_i = f_l(x_i)$$, where

$$f_l = \arg\min_{f}\sum_{i=1}^n {(y_i-f(x_i))}^2. \qquad\text{(2)}$$

The proof that I think requires the least work is as follows:

If $$\lambda=0$$ then (1) and (2) are the same. Further (1) is continuous in $$\lambda$$, therefore you can write

$$f_s(x,\lambda) = f_l(x) + r(x,\lambda),$$

where $$r(x,0)=0$$ and $$r$$ is continuous in $$\lambda$$.

• But shouldn't $\hat y_i$ in the local regression consider some neighbourhood of points, not just mean? And it would be very helpful if you wrote in matrix form. – Bruh Mar 8 at 16:01
• @Bruh see my comment to your question. I need a lot more information if you want a more specific answer. What do you mean by "local regression part"? That kind of implies that you are thinking about a specific smoothing spline. Give those specifics. – Sebastiaan Mar 9 at 15:16