I wish to generate confidence intervals for a smoothing spline using the pointwise standard error of $\hat{f}_\lambda(x)$. In particular, I am trying to construct the following interval: $$\hat{f}_\lambda(x) \pm 2\cdot se(\hat{f}_\lambda(x))$$

Now I know that in the Linear Regression setting where $y = x_0^T\beta+\epsilon$ we find that $var(x_0^T\hat{\beta}) = x_0^T(X^TX)^{-1}x_0\sigma^2$ and the confidence interval is given by: $$\hat{y}_0\pm t_{n-p}^{(\alpha/2)}\hat{\sigma}\sqrt{x_0^T(X^TX)^{-1}x_0 + 1}$$

In order to do this find an equivalent expression in the smoothing spline setting I need to find an expression for $Var(\hat{f}_\lambda(x_0))$: $$Var(\hat{f}_\lambda(x_0)) = Var(n_0^T(N^TN + \lambda \Omega_N)^{-1}N^Ty) $$ (where $n_0$ denotes the appropriate basis expansion of $x_0$) $$= n_0^T(N^TN + \lambda \Omega_N)^{-1}N^T Var(y) (n_0^T(N^TN + \lambda \Omega_N)^{-1}N^T)^T$$ $$= \sigma^2 \cdot n_0^T(N^TN + \lambda \Omega_N)^{-1}N^T(n_0^T(N^TN + \lambda \Omega_N)^{-1}N^T)^T$$

Now according to Green & Silverman (1994) and Wahba (1990) we can estimate $\sigma$ as:

$$\hat{\sigma}^2 = \frac{RSS(\hat{\lambda})}{Trace(\mathbb{1} - S_\lambda )}$$

which leaves us with the following interval:

$$= n_0^T(N^TN + \lambda \Omega_N)^{-1}N^Ty \pm 2\hat{\sigma} \cdot \sqrt{n_0^T(N^TN + \lambda \Omega_N)^{-1}N^T(n_0^T(N^TN + \lambda \Omega_N)^{-1}N^T)^T + 1}$$

I suspect this not to be correct so my question is: What is pointwise standard error and confidence interval for a smoothing spline?

For further details on my motivation to find these intervals see Figure 5.9 in The Elements of Statistical Learning (Hastie et. al.).

  • 1
    $\begingroup$ In the most popular package, MGCV, error bands work like this. For self-study (?) the theory is described here (assuming a given fixed $\lambda$). You are correct that estimating residual variance is key. $\endgroup$
    – GeoMatt22
    Commented Mar 11, 2020 at 3:12
  • $\begingroup$ Thanks @GeoMatt22 , maybe I used the self-study tag incorrectly - happy to see a full answer. That link is useful, I will try and figure out how to implement it and update the question here. $\endgroup$
    – Seraf Fej
    Commented Mar 11, 2020 at 15:21


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.