# Are LOESS and GAM with one covariate the same?

I am learning about Generalized Additive Models (GAMS) and have found in multiple locations a claim that a GAM with one independent variable is identical to a bivariate LOESS regression. Is this claim correct?

I have been unable to obtain the same answer using the loess function in base R and the gam function in the R package mgcv. Here is a link to a similar question from 2009 on a different forum: http://r.789695.n4.nabble.com/Difference-between-gam-and-loess-td881045.html

Below is my R code, which I modified from code provided at the link above. The gam function in R has numerous options. If a GAM with one independent variable is identical to a bivariate LOESS regression could someone provide the option settings needed? I have tried a wide range of values for span in the loess function and am only showing a few.

Thank you for any advice or assistance.

library(mgcv)

set.seed(1234)

# generate data
x <- sort(runif(100))
y <- sin(2*pi*x) + rnorm(10, sd=0.1)

mgcv.1 <- gam(y ~ s(x), family=gaussian(), weights=NULL, subset=NULL,
offset=NULL, method = "GCV.Cp",
optimizer=c("outer", "newton"), control=list(), scale=0,
select=FALSE, knots=NULL, sp=NULL, min.sp=NULL, H=NULL, gamma=1,
fit=TRUE, paraPen=NULL, G=NULL, drop.unused.levels=TRUE,
bs="cr")

base.r <- loess(y ~ x, degree=1, span=0.50) ; summary(base.r$fitted - mgcv.1$fitted)
base.r <- loess(y ~ x, degree=1, span=0.75) ; summary(base.r$fitted - mgcv.1$fitted)
base.r <- loess(y ~ x, degree=2, span=0.50) ; summary(base.r$fitted - mgcv.1$fitted)
base.r <- loess(y ~ x, degree=2, span=0.75) ; summary(base.r$fitted - mgcv.1$fitted)


EDIT

According to lecture notes at the link here:

http://polisci.msu.edu/jacoby/icpsr/regress3/lectures/week4/16.GAM.pdf

the gam function in mgcv only fits smoothing splines, but local polynomial regression could be done in S-PLUS. Perhaps that is why I cannot obtain the same answer with mgcv as I get with the loess function? Perhaps another GAM package in R does allow local polynomial regression?

• see my answer below (gam), your edit and the answer seem to be saying pretty much the same thing. – Christoph Hanck Jul 9 '15 at 15:49
• @ChristophHanck Thank you, Christoph. I will explore the gam package some more, as well as the car package and possibly others. – Mark Miller Jul 9 '15 at 15:53

Not really a full answer, but too long for a comment: s sets up a spline, whereas loess does a local regression.

In the gam package (maybe mgcv too, not too familiar with that one) you can also feed a local regression, as in

library(gam)

set.seed(1234)

# generate data
x <- sort(runif(100))
y <- sin(2*pi*x) + rnorm(10, sd=0.1)

gam.1 <- gam(y ~ lo(x))
base.r <- loess(y ~ x)
summary(base.r$fitted - gam.1$fitted)
plot(base.r$fitted,gam.1$fitted)


That does not produce the same fitted values either, but maybe you can further play around with the settings of lo and loess.

• If you specify the degree and span, they are identical. gam.1 <- gam(y ~ lo(x, degree=2)) base.r <- loess(y ~ x, span=0.5, degree=2) – Eli Holmes Nov 21 '20 at 17:06

"LOESS" uses local kernel regression but is not a pure local kernel regression.

Local regression for a pre-specified bandwidth or pre-specified set of varying bandwidths can be written as a linear function of the data.

LOESS is however, non-linear, in that it attempts to introduce a degree of "robustification" to outliers (by downweighting large residuals and refitting).

As a result, in the general case, LOESS results won't be exactly reproduced by playing about with the settings for local linear regression, though if the data are sufficiently "nice" there may be a correspondence if the same settings are used -- LOESS also uses a "span" parameter that alters bandwidth to "cover" a given fraction of the data; even without any issues with large residuals, a local regression method would have to adjust bandwidth in the same way to reproduce it.

If your link function is identity (i.e., the error's PDF is Gaussian), a one covariate GAM is nothing else than the smooth version of your scatter plot. And this is generally a locally weighted scatterplot smoother. Read Hastie and Tibshirani 1986, particularly their section 5.2: They fit the GAMs by Fisher local scoring where the weighted least square fit is substituted by the more general (local) smoothing. Although they do not call it a LOESS, they speak about a running lines smoother with weights (dp/dq)^2*V^(-1)), which is basically a local weighted smoother. If your link function is identity the scoring procedure has no iterations and the linear estimator eta reduces to the smooth of the scatter plot.

Why R does not reflect this behaviour I do not know (I know little about R). I guess you have to specify in the LOESS function that your weights are all 1s (otherwise, I think they depend automatically on the distance between the observation points) and/or surely you have to use the same span in both GAM and LOESS.