# GAMs vs GLMs with feature engineering - is there a practical difference?

I recently came across this tutorial on General Additive Models (GAMs). Quoting the article:

The principle behind GAMs is similar to that of regression, except that instead of summing effects of individual predictors, GAMs are a sum of smooth functions. Functions allow us to model more complex patterns, and they can be averaged to obtain smoothed curves that are more generalizable.

Is a GLM with smooth predictors (i.e., independent variables) practically any different from a GLM with smooth functions? Does smoothness actually matter?

The most competitive data scientists (e.g., those on Kaggle) say that when everyone is competing with the same modeling packages, what sets the best data scientists apart from the rest is the ability to do feature engineering well. Quoting this article:

With the extensive amount of free tools and libraries available for data analysis, everybody has the possibility of trying out advanced statistical models in a competition. As a consequence of this, what gives you most “bang for the buck” is rarely the statistical method you apply, but rather the features you apply it to. By feature engineering, I mean using domain specific knowledge or automatic methods for generating, extracting, removing or altering features in the data set.

Back to the original article, the term "function" sounds vague. To someone without applied GAM experience, such as myself, "function" sounds like a synonym for a predictor that has simply gone through feature engineering or transformation. This leads me to believe that if I am already doing feature engineering (and if I'm also using GLMs) for a predictive model, then I likely won't get any additional predictive benefit using GAMs. Is that a fair assumption?

• What do you mean by "smooth predictors" in your second sentence? In a sense you are doing some aspects of feature engineering with a GAM. Fitting a GAM would be similar to including $x$, $x^2$, and $x^3$, exact than instead of representing the effect of $x$ as a set of global polynomial basis functions, the GAM uses local spline basis functions of some description. I usually takle feature engineering to mean much more than this though. GAMs are only useful is the truth is smooth or a smooth function is a close approximation to the the truth. If the truth is not smooth other features are needed Sep 13, 2017 at 15:46
• @GavinSimpson I mean smooth only in the sense that GAM functions are smooth, as mentioned in the first link. So is it fair to say that if all of my feature engineering were simply polynomial transformations (generally not the case, but putting that aside for the moment), then the GAM and GLM would probably output similar predictions? Sep 13, 2017 at 16:34
• Yes; a GAM is just a GLM but using a spline basis expansion for one or more of the covariates. If you took the same basis expansion and used it in the GLM you'd get the same model (for fixed degree of smoothness). The main difference comes if you want to do smoothness selection (decide how wiggly any smooths should be) as the GAM then becomes a penalized GLM and the IRLS algorithm becomes a penalized IRLS algorithm. If you fix the wiggliness, then GAM == GLM. Sep 13, 2017 at 17:13