26
$\begingroup$

I wish to better understand the pros/cons for using either loess or a smoothing splines for smoothing some curve.

Another variation of my question is if there is a way to construct a smoothing spline in a way that will yield the same results as using loess.

Any reference or insight are welcomed.

$\endgroup$
  • $\begingroup$ Tal, The following well-cited article looks at many non-parametric regression approaches Buja, A., Hastie, T., & Tibshirani, R. (1989). Linear Smoothers and Additive Models. The Annals of Statistics, 17(2), 453–510, best. $\endgroup$ – Alexis Mar 27 '19 at 21:30
16
$\begingroup$

Here is some R code/example that will let you compare the fits for a loess fit and a spline fit:

library(TeachingDemos)
library(splines)

tmpfun <- function(x,y,span=.75,df=3) {
    plot(x,y)
    fit1 <- lm(y ~ ns(x,df))
    xx <- seq( min(x), max(x), length.out=250 )
    yy <- predict(fit1, data.frame(x=xx))
    lines(xx,yy, col='blue')
    fit2 <- loess(y~x, span=span)
    yy <- predict(fit2, data.frame(x=xx))
    lines(xx,yy, col='green')
    invisible(NULL)
}

tmplst <- list( 
    span=list('slider', from=0.1, to=1.5, resolution=0.05, init=0.75),
    df=list('slider', from=3, to=25, resolution=1, init=3))

tkexamp( tmpfun(ethanol$E, ethanol$NOx), tmplst )

You can try it with your data and change the code to try other types or options. You may also want to look at the loess.demo function in the TeachingDemos package for a better understanding of what the loess algorythm does. Note that what you see from loess is often a combination of loess with a second interpolation smoothing (sometimes itself a spline), the loess.demo function actually shows both the smoothed and the raw loess fit.

Theoretically you can always find a spline that approximates another continuous function as close as you want, but it is unlikely that there will be a simple choice of knots that will reliably give a close approximation to a loess fit for any data set.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Hi Greg: 1) Thank you for the answer. 2) I love your loess.demo function... $\endgroup$ – Tal Galili Dec 20 '11 at 17:17
31
$\begingroup$

The actual results from a smoothing spline or loess are going to be pretty similar. They might look a little different at the edges of the support, but as long as you make sure it's a "natural" smoothing spline they will look really similar.

If you are just using one to add a "smoother" to a scatterplot, there's no real reason to prefer one over the other. If instead you want to make predictions on new data, it's generally much easier to use a smoothing spline. This is because the smoothing spline is a direct basis expansion of the original data; if you used 100 knots to make it that means you created ~100 new variables from the original variable. Loess instead just estimates the response at all the values experienced (or a stratified subset for large data).

In general, there are established algorithms to optimize the penalty value for smoothing splines (mgcv in R probably does this the best). Loess isn't quite as clear cut, but you'll generally still get reasonable output from any implementation. MGCV also gives you a feel for equivalent Degrees of Freedom so you can get a feel for how "non-linear" your data is.

I find that when modeling on very large data, a simpler natural spline often provides similar results for minimal calculation compared to either a smoothing spline or loess.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ +1, nice answer! I like the clear theoretical exposition. $\endgroup$ – gung - Reinstate Monica Dec 23 '11 at 4:57
  • $\begingroup$ Why do they differ near the edges of the support though? $\endgroup$ – imu96 Mar 6 '19 at 10:56
  • 1
    $\begingroup$ @imu96 AFAIK, non-parametric regression methods fall down at either end because they rely on a neighborhood of data to either side of the point along the $x$ axis being estimated, and towards the edges there is data on only one side, so the CIs get super wide, and the estimates get a tad biased. Each method suffers in this regard in a different manner because of different weights, different ways of defining neighborhood of data, etc. $\endgroup$ – Alexis Mar 27 '19 at 21:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.