12
$\begingroup$

Is it wrong that splines are only available in GAM-models, and not in GLM-models? I heard this a while back, and wonder if this is just a misconception, or has some truth to it. Here is an illustration: http://www.stats.uwo.ca/faculty/bellhouse/glm%20and%20gam.pdf

$\endgroup$
18
$\begingroup$

You are mistaken. Splines have a linear representation using derived covariates. As an example, a quadratic trend is non-linear, but can be modeled in a linear model by taking: $E[Y|X] = \beta_0 + \beta_1 X + \beta_2 X^2$, thus $X$ and its square are input into a linear model.

The spline can simply be seen as a sophisticated parametrization of one or more continuously or pseudo-continuously valued covariates.

$\endgroup$
  • $\begingroup$ Thank you for answering! So by saying that I am mistaken, you mean that splines can be used in GLM, correct? Did not completely understand. $\endgroup$ – HeyJane Aug 19 '17 at 19:21
  • $\begingroup$ Yes absolutely. In R, import the package splines, and running bs(...) allows you to create a linear representation of a spline with a user-specified polynomial degree and knot-points. $\endgroup$ – AdamO Aug 19 '17 at 19:32
  • 7
    $\begingroup$ I wrote a lot about this question here: madrury.github.io/jekyll/update/statistics/2017/08/04/… $\endgroup$ – Matthew Drury Aug 19 '17 at 19:33
  • $\begingroup$ Thank you very much to the both of you! I see it now, AdamO! Great page, Matthew, I'll read it all! :) $\endgroup$ – HeyJane Aug 19 '17 at 19:43
12
$\begingroup$

@AdamO's answer is correct, in that spline-based fits can certainly be done in the standard GLM framework. That's not to say that GAM's are just a special case of GLM's though! While there are a series of models that exactly identical and can be framed as both a GAM or as a GLM with a spline expansion of the covariates, there are some GAM models that are not available in the standard GLM framework.

For example, one could fit a GAM model using a smoothing spline for each of the covariates. This basically results in a spline expansion of the variables, but with a penalty on the second derivatives. This results in a model that is a bit outside the standard GLM framework.

In addition, it's often considered standard procedure, and is built into most GAM libraries, to fit smoothing parameters (i.e. spline degrees of freedom, etc.) by optimizing various measures of out of sample errors, while the GLM formulation typically considers the covariate space fixed.

$\endgroup$
  • $\begingroup$ I wish I could upvote you, but I do not have enough points. Thank you for contributing. I'm not sure I understand your second paragraph: you are saying that smoothing splines can only be fit with GAM? Could you elaborate what exactly is the difference between a regular cubic spline and a smoothing cubic spline? I understand this is much to ask. $\endgroup$ – HeyJane Aug 19 '17 at 20:45
  • $\begingroup$ @HeyJane: if you look at the wikipedia page, you will note that these splines are penalized by their second derivative. This allows one to control the smoothness by a continuous penalty rather than an integer degrees of freedom. As such, it is a penalized maximum likelihood problem, rather than standard maximum likelihood problem. This means you can't fit them directly with R's glm function, unlike when using standard cubic splines with a glm. $\endgroup$ – Cliff AB Aug 19 '17 at 21:15
  • 2
    $\begingroup$ Aha! I get it! So instead of, with a regular cubic spline, saying that we just want the second derivatives to be equal at the knots, we want to impose some property on the second derivative, i.e. second derivative not being to high, hence the penalty term? $\endgroup$ – HeyJane Aug 19 '17 at 21:23
  • $\begingroup$ @HeyJane: yes, I would say that's a good summary. $\endgroup$ – Cliff AB Aug 19 '17 at 21:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.