I wish to better understand the pros/cons for using either loess or a smoothing splines for smoothing some curve.
Another variation of my question is if there is a way to construct a smoothing spline in a way that will yield the same results as using loess.
Any reference or insight are welcomed.
Here is some R code/example that will let you compare the fits for a loess fit and a spline fit:
library(TeachingDemos)
library(splines)
tmpfun <- function(x,y,span=.75,df=3) {
plot(x,y)
fit1 <- lm(y ~ ns(x,df))
xx <- seq( min(x), max(x), length.out=250 )
yy <- predict(fit1, data.frame(x=xx))
lines(xx,yy, col='blue')
fit2 <- loess(y~x, span=span)
yy <- predict(fit2, data.frame(x=xx))
lines(xx,yy, col='green')
invisible(NULL)
}
tmplst <- list(
span=list('slider', from=0.1, to=1.5, resolution=0.05, init=0.75),
df=list('slider', from=3, to=25, resolution=1, init=3))
tkexamp( tmpfun(ethanol$E, ethanol$NOx), tmplst )
You can try it with your data and change the code to try other types or options. You may also want to look at the loess.demo function in the TeachingDemos package for a better understanding of what the loess algorythm does. Note that what you see from loess is often a combination of loess with a second interpolation smoothing (sometimes itself a spline), the loess.demo function actually shows both the smoothed and the raw loess fit.
Theoretically you can always find a spline that approximates another continuous function as close as you want, but it is unlikely that there will be a simple choice of knots that will reliably give a close approximation to a loess fit for any data set.
The actual results from a smoothing spline or loess are going to be pretty similar. They might look a little different at the edges of the support, but as long as you make sure it's a "natural" smoothing spline they will look really similar.
If you are just using one to add a "smoother" to a scatterplot, there's no real reason to prefer one over the other. If instead you want to make predictions on new data, it's generally much easier to use a smoothing spline. This is because the smoothing spline is a direct basis expansion of the original data; if you used 100 knots to make it that means you created ~100 new variables from the original variable. Loess instead just estimates the response at all the values experienced (or a stratified subset for large data).
In general, there are established algorithms to optimize the penalty value for smoothing splines (mgcv in R probably does this the best). Loess isn't quite as clear cut, but you'll generally still get reasonable output from any implementation. MGCV also gives you a feel for equivalent Degrees of Freedom so you can get a feel for how "non-linear" your data is.
I find that when modeling on very large data, a simpler natural spline often provides similar results for minimal calculation compared to either a smoothing spline or loess.
