# How to calculate Hat matrix for penalized spline regressions?

The book "Semiparametric Regression" by Ruppert et al. (2003) provided a computationally fast algorithm for Penalized Spline Regression. I put a part of the algorithm here. Does anybody can do algebra to prove equation that is mentioned on step 4? Any help would be much appreciated.

I get an ever so slightly different formula, but it only differs in where a transpose is placed.

Let's start with the expression $$C^t C + \alpha D$$ and use the hypothesized decomposition $$C^t C = R^t R$$ to write:

\begin{align*} C^t C + \alpha D &= R^t R + \alpha D \\ &= R^t R + \alpha R^{t} R^{-t} D R^{-1} R \\ &= R^t ( I + \alpha R^{-t} D R^{-1} ) R \\ &= R^t ( I + \alpha U \text{diag}(s) U^t ) R \end{align*}

In the last step we've used $R^{-t} D R^{-1} = U \text{diag}(s) U^t$, another hypothesis. Note that $U^t U = I$, since $U$ is an orthogonal matrix.

Breaking down the inverse:

\begin{align*} ( I + \alpha U \text{diag}(s) U^t )^{-1} &= ( U U^t + \alpha U \text{diag}(s) U^t ) \\ &= ( U (I + \alpha \ \text{diag}(s)) U^t)^{-1} \\ &= U^{-t} ( I + \alpha \ \text{diag}(s) )^{-1} U^{-1} \end{align*}

Now the matrix $I + \alpha \ \text{diag}(s)$ is itself diagonal, so its inverse is as well. We can write its inverse conveniently as

$$\frac{1}{1 + \alpha \ \text{diag}(s)}$$

Now put it all together to get:

\begin{align*} C ( C^t C + \alpha D )^{-1} C^t y &= C R^{-1} U^{-t} \frac{1}{1 + \alpha \ \text{diag}(s)} U^{-1} R^{-t} C^{t} y \\ &= C R^{-1} U^{-t} \frac{1}{1 + \alpha \ \text{diag}(s)} ( C R^{-1} U^{-t} )^t y \\ &= A \frac{1}{1 + \alpha \ \text{diag}(s)} A^{t} y \end{align*}

which, up to a substitution of $U^t$ with $U$, is the formula quoted.

• @A.Gh You're very welcome, I quite like this subject. I'd much appreciate it if you accepted my answer if it was what you were after. Thanks. – Matthew Drury Jul 15 '15 at 21:34