3
$\begingroup$

I read a paper on thin plate splines that stated several advantages. Thin plate splines:

  1. Do not rely on ’knots’ as many other spline methods do

  2. Are of isotropic character (i.e. unaffected by the rotation of the coordinate system of the predictors)

  3. Are ’low rank’, which means that "they have fewer coefficients than there are data to smooth"

I don't understand no.3. What do they mean by "low rank" and "fewer coefficients than there are data to smooth"?

$\endgroup$

1 Answer 1

5
$\begingroup$

A thin plate spline has a knot for every unique value of the input covariate, or unique combination of values of the input covariates if doing a 2+d smooth. This is why thin plate splines do away with the issue of knot placement and the choice as to how many knots to use; there are as many knots as unique values of the covariate. Job done.

Except that this results in an exceedingly large and rich basis, one which is so rich in fact that i) it is unlikely we want to fit functions of such complexity that require one basis function per unique value of the covariates, and ii) it will slow down model fitting because we're carrying around such a large basis and penalty matrix needlessly.

Basically we don't need all the information in the thin plate spline basis for the sorts of models envisioned by GAMs. In fact we probably only need a small fraction of the information contained in the basis.

What Simon Wood, author of mgcv, showed (Wood, 2003) was that if you eigendecompose the full thin plate regression spline (TPRS) basis and take the first k eigenvectors as a new basis, you concentrate a lot of the signal in the original basis in the new one, whilst drastically reducing the size of the basis needed to fit the model.

This is what is meant by "low rank"; the model is not using the full rank (all of the columns) of the TPRS basis but rather a low rank representation (approximation) of the full basis. It is low rank because it has lower dimensionality than the full TPRS basis. Because each column in the basis used to fit the model is associated with a model coefficient, the low rank TPRS basis requires many fewer coefficients to be estimated than the full TPRS basis.


Wood, S.N., 2003. Thin plate regression splines. J. R. Stat. Soc. Series B Stat. Methodol. 65, 95–114. https://doi.org/10.1111/1467-9868.00374

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.