Here are 2 questions. Your expertise on those issues would be highly appreciated.

In the GAM approach, it makes sense to start with a highly flexible approach and then apply penalties to achieve the smoothness required for a plausible shape. While fitting GAMs, I always use P-spline (=penalized B-splines). However, S. Wood recommend to use the penalized thin-plate spline as it tends to give the best MSE performance.

1) There are plenty of different types of smoothing splines. Why should we use other smoothing splines than the penalized thin-plate spline as it is the one that gives best MSE performance?

2) S. Wood writes:

"Broadly speaking the default penalized thin plate regression splines tend to give the best MSE performance, but they are slower to set up than the other bases. The knot based penalized cubic regression splines (with derivative based penalties) usually come next in MSE performance, with the P-splines doing just a little worse. However the P-splines are useful in non-standard situations".

For P-splines, what does "non-standard situations" mean? When P-splines can be preferred than penalized thin-plate spline?

Wood, S. N. (2003). Thin plate regression splines. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 65(1):95–114.

  • $\begingroup$ Please note the caveat in the statement: "[...] tend [to give best MSE...]" $\endgroup$
    – Jim
    Jun 17 '18 at 9:45

Question 1:

Why would you not want to use thin plate splines?

  1. It is computationally costly to set up the basis functions for a thin plate spline. Typically you would need one basis function per data point; even though Simon's truncation process allows for far fewer basis functions to be used in fitting, you still need to create all the basis functions before doing the truncation/decomposition of them.

  2. There are settings where thin plate splines are not optimal. Smooths on finite areas with potentially irregular shapes are one such instance. Thin plate splines do not respect boundaries of the domain over which a smooth is required. For example:

enter image description here

A thin plate spline would smooth through/across the boundary between the two arms of the rotated U; the low values of the lower arm of the U would leak across the boundary and vice versa, producing biased estimated values of the function as it approaches the boundary.

Question 2:

P splines are useful in several non-standard settings. For example

  1. They are often used as the basis for adaptive splines, where the degree of wiggliness of $f(x)$ is allowed to vary as a smooth function of $x$.

  2. Shape constrained splines, such as splines constrained to be monotonic increasing, are quite easily built from modifications of P spline bases.

  • 1
    $\begingroup$ Many thanks for your clear responses. One last precision. What about the sample size required for fitting GAM. Is there a kind of rule of thumb because with GAM if there is not enough data GAM will penalize to linear. $\endgroup$ Mar 24 '18 at 22:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.