7
$\begingroup$

Here are 2 questions. Your expertise on those issues would be highly appreciated.

In the GAM approach, it makes sense to start with a highly flexible approach and then apply penalties to achieve the smoothness required for a plausible shape. While fitting GAMs, I always use P-spline (=penalized B-splines). However, S. Wood recommend to use the penalized thin-plate spline as it tends to give the best MSE performance.

1) There are plenty of different types of smoothing splines. Why should we use other smoothing splines than the penalized thin-plate spline as it is the one that gives best MSE performance?

2) S. Wood writes:

"Broadly speaking the default penalized thin plate regression splines tend to give the best MSE performance, but they are slower to set up than the other bases. The knot based penalized cubic regression splines (with derivative based penalties) usually come next in MSE performance, with the P-splines doing just a little worse. However the P-splines are useful in non-standard situations".

For P-splines, what does "non-standard situations" mean? When P-splines can be preferred than penalized thin-plate spline?

Wood, S. N. (2003). Thin plate regression splines. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 65(1):95–114.

$\endgroup$
1
  • $\begingroup$ Please note the caveat in the statement: "[...] tend [to give best MSE...]" $\endgroup$
    – Jim
    Jun 17, 2018 at 9:45

1 Answer 1

1
$\begingroup$

Question 1:

Why would you not want to use thin plate splines?

  1. It is computationally costly to set up the basis functions for a thin plate spline. Typically you would need one basis function per (unique) data point; even though Simon's truncation process allows for far fewer basis functions to be used in fitting, you still need to create all the basis functions before doing the truncation/decomposition of them, which is also a costly eigendecomposition.

  2. There are settings where thin plate splines are not optimal. Smooths on finite areas with potentially irregular shapes are one such instance. Thin plate splines do not respect boundaries of the domain over which a smooth is required. For example:

enter image description here

A thin plate spline would smooth through/across the boundary between the two arms of the rotated U; the low values of the lower arm of the U would leak across the boundary and vice versa, producing biased estimated values of the function as it approaches the boundary.

Question 2:

P splines are useful in several non-standard settings. For example

  1. They are often used as the basis for adaptive splines, where the degree of wiggliness of $f(x)$ is allowed to vary as a smooth function of $x$.

  2. Shape constrained splines, such as splines constrained to be monotonic increasing, are quite easily built from modifications of P spline bases.

$\endgroup$
7
  • 1
    $\begingroup$ Many thanks for your clear responses. One last precision. What about the sample size required for fitting GAM. Is there a kind of rule of thumb because with GAM if there is not enough data GAM will penalize to linear. $\endgroup$ Mar 24, 2018 at 22:50
  • $\begingroup$ @Gavin Simpson If thin-plate splines use the same number of unique observations as the knots and typically need one basis function for each data point then isn't this restrict the size of the basis function $K$? $\endgroup$
    – shani
    Mar 28 at 4:46
  • $\begingroup$ @shani I'm not sure what your question is. The TPS has one basis function per unique value of the covariate. It is a very rare applied situation where we would need that many basis functions to fit the true but unknown function. But there is no more restriction on k than there is for a CRS for example; you can't fit the GAM with more coefficients (i.e. more basis functions) than you have unique data. $\endgroup$ Mar 28 at 6:17
  • 1
    $\begingroup$ @shani, yes for the TPS, but {mgcv} uses a low-rank TPR, not the full TPS. In that construction $K \ll n$ (to use your notation). This is not without computational cost; forming the low-rank version involves finding the first $K$ eigenvectors of the full TPS basis, which is what I am going on about in Q1.1. If your point is that I said "...one basis function per data point" then OK (I was being unnecessarily loose with my terminology), but why not just say that? $\endgroup$ Mar 28 at 9:11
  • 1
    $\begingroup$ @shani as for the other question, I don't know exactly why, but it is a restriction in all the GAM software that I have used. $\endgroup$ Mar 28 at 9:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.