# What are the differences between generalized additive model, basis expansion and boosting?

I am confused with the term

• Basis expansion
• Boosting

If we fit a data with "spline basis", is it a "Generalized additive model"? To me it is just a linear model with different basis, we can do it with polynomial basis or Fourier basis etc.

Also there is a notion "additive" in "Generalized additive model" how it different from boosting?

• This is a nice question which is a bit hard to make short. In case you are not already aware of the package mboost has a gamboost function that explicitly interweaves GAMs and Gradient Boosting. To state the obvious: the final estimate from a GBM can be interpreted as an additive prediction function. The whole issue in on the update schedule and on the nature of the base learners used. – usεr11852 Aug 13 '17 at 11:48
• @usεr11852 thanks for the comment, what do you think about Carl's answer? – Haitao Du Aug 14 '17 at 13:33
• I have to read a bit more material to assess it properly. It seems correct though. Some of the references mentioned I have read a while back (K&V 88,89) and I can't remember their direct implications. +1 though, it is obviously useful. – usεr11852 Aug 14 '17 at 17:40

Basis expansion implies a basis function. In mathematics, a basis function is an element of a particular basis for a function space. For example, sines and cosines form a basis for Fourier analysis and can duplicate any waveform shape (square waves, sawtooth waves, etc.) just by adding enough basis functions together. From Basis (linear aglebra) "In mathematics, a set of elements (vectors) in a vector space V is called a basis, or a set of basis vectors, if the vectors are linearly independent and every vector in the vector space is a linear combination of this set." The object of finding basis functions is to create a spanning set. For example, "The real vector space R$^3$ has {(-1,0,0), (0,1,0), (0,0,1)} as a spanning set. This particular spanning set is also a basis. If (-1,0,0) were replaced by (1,0,0), it would also form the canonical basis of R$^3$."