# What are the differences between generalized additive model, basis expansion and boosting?

I am confused with the term

• Generalized additive model
• Basis expansion
• Boosting

If we fit a data with "spline basis", is it a "Generalized additive model"? To me it is just a linear model with different basis, we can do it with polynomial basis or Fourier basis etc.

Also there is a notion "additive" in "Generalized additive model" how it different from boosting?

• This is a nice question which is a bit hard to make short. In case you are not already aware of the package mboost has a gamboost function that explicitly interweaves GAMs and Gradient Boosting. To state the obvious: the final estimate from a GBM can be interpreted as an additive prediction function. The whole issue in on the update schedule and on the nature of the base learners used. Aug 13, 2017 at 11:48
• @usεr11852 thanks for the comment, what do you think about Carl's answer? Aug 14, 2017 at 13:33
• I have to read a bit more material to assess it properly. It seems correct though. Some of the references mentioned I have read a while back (K&V 88,89) and I can't remember their direct implications. +1 though, it is obviously useful. Aug 14, 2017 at 17:40

Basis expansion implies a basis function. In mathematics, a basis function is an element of a particular basis for a function space. For example, sines and cosines form a basis for Fourier analysis and can duplicate any waveform shape (square waves, sawtooth waves, etc.) just by adding enough basis functions together. From Basis (linear aglebra) "In mathematics, a set of elements (vectors) in a vector space V is called a basis, or a set of basis vectors, if the vectors are linearly independent and every vector in the vector space is a linear combination of this set." The object of finding basis functions is to create a spanning set. For example, "The real vector space R$^3$ has {(-1,0,0), (0,1,0), (0,0,1)} as a spanning set. This particular spanning set is also a basis. If (-1,0,0) were replaced by (1,0,0), it would also form the canonical basis of R$^3$."