2
$\begingroup$

I do not understand the relationship between knots and basis functions in Generalized Additive Models (hereafter GAM).

In Chapter 4 of S. Wood's book "Generalized Additive Models - An Introduction with R", Wood uses polynomials to explain basis functions. If I understand correctly, polynomial models qualify as GAM, but they tend to perform poorly due to their excessive "wiggliness". To obviate excessive "wiggliness", tent functions can be used, which are only allowed to assume non-zero values nearby the knots. A GAM based on tent basis functions is a piecewise linear regression.

One way to achieve a GAM that is not too "stiff" (like a piecewise linear regression) and not too "wiggly" (like a polynomial) is to use cubic splines as basis functions.

Am I correct in saying that, in such scenario, the n-th basis function is a cubic spline defined so that it can assume non-zero values in proximity of the n-th knot, while it should tend to zero everywhere else, similarly to how a tent function behaves?

If the statement above is true, how comes that in the following example, a GAM fitted using five knots is based on only four basis functions, and some of the basis functions assume values that are far from zero in regions that are not near any knots?

# Example in R
# generate data with a sinusoid dependence from the explanatory variable x:
set.seed(1)
x <- seq(0, pi * 2, 0.1)
sin_x <- 3 + sin(x)
y <- sin_x + rnorm(n = length(x), mean = 0, sd = sd(sin_x / 2))
Sample_data <- data.frame(y,x)

# fit GAM
gam_y <- mgcv::gam(y ~ s(x, bs="cr", k=5),
                    knots = list(x = c(1:5)),
                    method = "REML")
# diagnostics look good:
# par(mfrow = c(2,2)); mgcv::gam.check(gam_y)

# plot data
plot(y~x, ylim=c(-2,5))

# plot GAM predictions as a 
gam_pred <- predict(gam_y, newdata = data.frame(x = Sample_data[,2]))
lines(x, gam_pred, col="red", lwd=2)

# display basis functions
model_matrix <- predict(gam_y, type = "lpmatrix")
matplot(x, model_matrix[,-1], type = "l", lty = 2, add = T)

# mark the x coordinates of the knots:
abline(v=c(1:5), col="grey", lty="dotted")

enter image description here

$\endgroup$
2
$\begingroup$

The basis for cubic regression splines that you use here can be found in Table 5.1 of Wood and is explained in Section 5.3.1. You can see that the constraints are on the first two derivatives and the value of the function at the knots, rather than whether or not the basis is non-zero in that area (whatever "area" means). Thinking about the shape of a cubic polynomial is useful for understanding the shape of the basis functions. Figure 5.1 provides a high-level summary of what the spline basis is achieving. Chapter 5 of Elements of Statistical Learning has more on splines. I found it to be a better explanation than Wood in this respect, but Wood is better for other reasons.

As for why there are 4 basis functions. I believe 1 is lost to an identifiability constraint that orthogonalizes the smooth to the intercept term. Again, Chapter 5 (in this case 5.4.1) has all your answers!

$\endgroup$
10
  • $\begingroup$ Thank you. Fig. 5.1 is indeed enlightening (almost as much as using polynomials as an example of GAMs, which was an eye-opener for me. Wood clearly knows his stuff). One thing that puzzles me is that a single cubic spline used to model the data directly can do an excellent job (E.g. see fig. 5.1), so I wonder what's the advantage of going through the bother of fitting a GAM that is a sum of n cubic splines as basis functions and, visually, it produces a very similar result. That's a topic for another stand-alone question. $\endgroup$ Jun 23 at 16:33
  • $\begingroup$ A single cubic spline is just a cubic polynomial, no? In that case all the problems associated with polynomials hold. $\endgroup$ Jun 23 at 16:37
  • $\begingroup$ As far as I understand, and according to the definitions I've seen so far, a spline is a piecewise polynomial itself. Cubic splines are not just one cubic polynomial, but a series of cubic polynomials "stitched together". See the fit of cube_bs_y <- lm(y ~ splines::bs(x, knots=c(1,2,3,4,5))), a single cubic spline, compared to gam_y as defined in the body of my question (i.e. a sum of four cubic splines). Am I off-mark? $\endgroup$ Jun 23 at 16:46
  • $\begingroup$ FYI: stats.stackexchange.com/questions/531564/… $\endgroup$ Jun 23 at 16:50
  • 1
    $\begingroup$ @MarcoPlebani Look at model.matrix(gam_y) & you'll see the columns in the model model matrix that are explicitly listed in the output from summary.lm. For most splines there's nothing of interest in the coefs for each basis function that make up a spline; the interest is on the estimated spline. As such summary.gam() focuses on the spline, summarizes it with the EDF for the spline plus a test against a null hypothesis of a constant zero function. I also think that the identifiability constraints are causing some confusion - the basis you plot doesn't look like a CRS basis usually does. $\endgroup$ Jun 28 at 6:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.