The book "Semiparametric Regression" by Ruppert et al. (2003) provided a computationally fast algorithm for Penalized Spline Regression. I put a part of the algorithm here. Does anybody can do algebra to prove equation that is mentioned on step 4? Any help would be much appreciated. enter image description here


I get an ever so slightly different formula, but it only differs in where a transpose is placed.

Let's start with the expression $$C^t C + \alpha D $$ and use the hypothesized decomposition $$C^t C = R^t R$$ to write:

$$\begin{align*} C^t C + \alpha D &= R^t R + \alpha D \\ &= R^t R + \alpha R^{t} R^{-t} D R^{-1} R \\ &= R^t ( I + \alpha R^{-t} D R^{-1} ) R \\ &= R^t ( I + \alpha U \text{diag}(s) U^t ) R \end{align*}$$

In the last step we've used $R^{-t} D R^{-1} = U \text{diag}(s) U^t$, another hypothesis. Note that $U^t U = I$, since $U$ is an orthogonal matrix.

Breaking down the inverse:

$$\begin{align*} ( I + \alpha U \text{diag}(s) U^t )^{-1} &= ( U U^t + \alpha U \text{diag}(s) U^t ) \\ &= ( U (I + \alpha \ \text{diag}(s)) U^t)^{-1} \\ &= U^{-t} ( I + \alpha \ \text{diag}(s) )^{-1} U^{-1} \end{align*}$$

Now the matrix $I + \alpha \ \text{diag}(s)$ is itself diagonal, so its inverse is as well. We can write its inverse conveniently as

$$ \frac{1}{1 + \alpha \ \text{diag}(s)} $$

Now put it all together to get:

$$\begin{align*} C ( C^t C + \alpha D )^{-1} C^t y &= C R^{-1} U^{-t} \frac{1}{1 + \alpha \ \text{diag}(s)} U^{-1} R^{-t} C^{t} y \\ &= C R^{-1} U^{-t} \frac{1}{1 + \alpha \ \text{diag}(s)} ( C R^{-1} U^{-t} )^t y \\ &= A \frac{1}{1 + \alpha \ \text{diag}(s)} A^{t} y \end{align*}$$

which, up to a substitution of $U^t$ with $U$, is the formula quoted.

  • $\begingroup$ @A.Gh You're very welcome, I quite like this subject. I'd much appreciate it if you accepted my answer if it was what you were after. Thanks. $\endgroup$ – Matthew Drury Jul 15 '15 at 21:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.