What is the difference between LOESS (locally estimated scatterplot smoothing) and LOWESS (locally weighted scatterplot smoothing)? From Wikipedia I can only see that LOESS is a generalization of LOWESS. Do they have slightly different parameters?
I think it is important to distinguish between methods and their implementations in software. The main difference with respect to the first is that lowess allows only one predictor, whereas loess can be used to smooth multivariate data into a kind of surface. It also gives you confidence intervals. In these senses, loess is a generalization. Both smooth by using tricube weighting around each point, and loess also adds an optional robustification option that re-weights residuals using biweight weighting.
Now for the implementation. In some software, lowess uses a linear polynomial, while loess uses a quadratic polynomial (though you can alter that). The defaults and shortcuts that the algorithms use are often quite different, so that it is hard to get the univariate outputs to match exactly. On the other hand, I am not aware of a case where the choice between the two made a substantive difference.
lowess and loess are algorithms and software programs created by William Cleveland. lowess is for adding a smooth curve to a scatterplot, i.e., for univariate smoothing. loess is for fitting a smooth surface to multivariate data.
Both algorithms use locally-weighted polynomial regression, usually with robustifying iterations. Local regression is a statistical method for fitting smoothing curves without prior assumptions about the shape or form of the curve.
There have been many implementations of Cleveland's approach, but the most well known and most used are probably the implementations in R.
The R core stats package contains
loess functions, both based quite closely on Cleveland's original code but with R interfaces.
lowess was published as a mathematical algorithm by Cleveland (1979) and as a Fortran software program by Cleveland (1981). lowess smoothing become popular when it was included as a function in the New S language in 1988 (Becker et al, 1988) and then later in R. The original Fortran program (dated 1985) is still available from Netlib (https://www.netlib.org/go/lowess). The R stats function is based on a translation of the Fortran program to C and was one of the earliest functions in R.
lowess function in R is designed for adding smooth curves to plots, so the output is just a list of ordered x coordinates and smoothed y values.
This style of output inputs easily into plotting functions in R:
x <- 1:1000 y <- rnorm(1000) plot(x, y) l <- lowess(x, y) lines(l)
The lowess method consists of computing a series of local linear regressions, with each local regression restricted to a window of x-values. Smoothness is achieved by using overlapping windows and by gradually down-weighting points in each regression according to their distance from the anchor point of the window (tri-cube weighting).
To conserve running time and memory, locally-weighted regressions are computed at only a limited number of anchor x-values, usually 100-200 distinct points.
Anchor points are at least
delta apart, where
delta is a parameter input to the algorithm.
The amount of smoothing is determined by the span of the overlapping windows, defined as a proportion of the total number of points.
The larger the span, the smoother the curve.
The span parameter is called
F in the Fortran code and
f in the R stats
For each anchor point, a weighted linear regression is performed for a window of neighboring points. The neighboring points consist of the smallest set of closest neighbors containing at least span proportion of all points. Each local regression produces a fitted value for that anchor point. Fitted values for other x-values are then obtained by linear interpolation between anchor points.
For the first iteration, the local linear regressions use tri-cube distance weights.
Subsequent iterations multiple the distance weights by robustifying weights.
Points with residuals greater than 6 times the median absolute residual are assigned weights of zero and otherwise Tukey's biweight function is applied to the residuals to obtain the robust weights.
More iterations produce greater robustness.
Cleveland originally suggested 2 robustifying iterations.
The default in the R stats
lowess function is 3.
loess was published as a mathematical model by Cleveland & Devlin (1988), as a Fortran routine by Cleveland & Grosse (1991) and as an S function by Cleveland, Grosse & Shyu (1992). The Netlib repository still contains Cleveland & Grosse's Fortran version from 1990 (http://www.netlib.org/a/loess) and both Fortran and C versions by Cleveland, Grosse & Shyu from 1992 (http://www.netlib.org/a/dloess). See cloess.pdf for extended documentation. Professor Brian Ripley ported Cleveland, Grosse & Shyu's code to R in 1998. The R function was originally part of Prof Ripley's modreg CRAN package and was merged into the stats package in 2003.
loess function in R is designed to behave like other regression fitting functions in R such as
It accepts a model formula:
fit.lo <- loess(y ~ x1 + x2)
and optional prior weights and outputs a fitted model object that can be input to standard R generic functions like
predict and so on.
The output may need a bit of post-processing however before it can be input to plotting functions like
Results are output in original data order rather than sorted by any of the x-variates.
loess is a direct generalization of
lowess in that locally weighted univariate regressions are simply replaced by locally weighted multiple regressions.
The implementation is more complicated however and it is harder to avoid consuming memory in the multivariate setup.
Comparing lowess to loess
When there is only one x-variable and no prior weights, the R functions
loess are in principle equivalent,
so it is possible to make direct comparisons between them.
loess has several capabilities that
It accepts a formula specifying the model rather than the x and y matrices
The model can include multiple predictors, factors and interactions.
It accepts prior weights.
It will estimate the "equivalent number of parameters" implied by the fitted curve.
On the other hand,
loess is much slower than
lowess and sometimes fails when
lowess succeeds, so both programs are kept in R.
The default settings of the two programs are very different. Here is an example in which I force
loess to do precisely the same numerical calculation:
> set.seed(2020617) > x <- 1:1000 > y <- rnorm(1000) > out.lowess <- lowess(x, y, f=0.3, iter=3, delta=0) > out.loess <- loess(y~x, span=0.3, degree=1, + family="symmetric", iterations=4, + surface="direct")
The smoothed values from the two functions are the same to machine precision:
> out.lowess$y[1:5]  0.04257433 0.04239860 0.04222676 0.04205882 0.04189474 > fitted(out.loess)[1:5]  0.04257433 0.04239860 0.04222676 0.04205882 0.04189474 > all.equal(out.lowess$y, fitted(out.loess))  TRUE
Things to note here:
loessdoes quadratic (degree=2) local regression by default instead of linear (degree=1)
Unless you specify
family="symmetric", loess will fit the curve by least squares, i.e., won't do any robustness iterations at all.
lowess and loess count iterations differently:
iterin lowess means the number of robustness iterations;
iterationsin loess means the total number of iterations including the least squares fit, i.e., iterations=iter+1
surface="direct"to force both functions to avoid interpolation by performing a local regression at every unique x-value.
Large data sets
The only aspect in which it is not possible to make
lowess agree exactly is in their treatment of large data sets. When x and y are very long, say 10s of thousands of observations, it is impractical and unnecessary to do the local regression calculation exactly, rather it is usual to interpolate between observations that are very close together. This interpolation is controlled by the
delta argument to
lowess and by the
surface arguments to
When there are a large number of observations,
lowess groups together those x-values that are closer than a certain distance apart. Although grouping observations based on distance is in principle the best approach, such an approach is impractical in the multivariate x-space that
loess is designed to deal with. So
loess instead groups observations together based on the number of observations on a cell rather than on distances. Because of this small difference,
loess will almost always give slightly different numerical results for large data sets.
lowess is in principle more accurate but the difference is generally small.
Where the difference between
loess becomes significant is in terms of speed and memory usage.
lowess remains fast and efficient even for very large datasets of millions of points.
On my laptop PC for example
lowess takes 3 seconds for a million points with the default span:
> x <- rnorm(1e6) > y <- rnorm(1e6) > system.time(l <- lowess(x, y)) user system elapsed 3.00 0.02 3.01
and only half a second if the span is reduced:
> system.time(l <- lowess(x, y, f=0.1)) user system elapsed 0.53 0.00 0.54
Such large datasets are impractical with
Other robust implementations
weightedLowess function of the
provides an implementation of lowess with the added ability
to accept prior weights.
weightedLowess provides a novel C translation of the original Fortran code and tries to preserve as many features of the original algorithm design as possible while generalizing the concept of "span" to account for the prior weights.
In principle, the
locfit.robust of the locfit CRAN package has the same functionality as
weightedLowess but in practice gives different and (to me) less accurate results.
Unfortunately, locfit has not been maintained by the original authors for a long time and contains bugs.
For example, any attempt to set the
iter argument to
locfit.robust leads to an error.
loessFit function of the limma package provides a consistent interface to
The SAS PROC LOESS provides an implementation of loess with the additional ability to estimate the span by minimizing the AICC criterion.
Other non-robust implementations
The following extend local-regression to more general contexts but omit the robustifying steps of the original algorithm.
The gam CRAN package provides the ability to include lowess curves in generalized linear model fits.
locfit functions of the locfit CRAN package extend the ideas of loess to generalized linear models and density estimation (Loader, 1999).
lowess function of Stata provides an implementation of univariate lowess.
It doesn't appear to implement the robustifying iterations and doesn't use interpolation, meaning that a local regression has to be fitted at every x-value.
lpoly function of Stata provides a flexible approach to multivariate locally-weighted polynomial smoothing with different choices of weights to the loess algorithm.
Cleveland, W.S. (1979). Robust Locally Weighted Regression and Smoothing Scatterplots. Journal of the American Statistical Association 74(368), 829-836.
Cleveland, W.S. (1981). LOWESS: A program for smoothing scatterplots by robust locally weighted regression. The American Statistician 35(1), 54.
Becker, R.A., Chambers, J.M. and Wilks, A.R. (1988). The New S Language: a Programming Environment for Data Analysis and Graphics. Wadsworth & Brooks/Cole, Pacific Grove.
Cleveland, W.S., and Devlin, S.J. (1988). Locally-weighted regression: an approach to regression analysis by local fitting. Journal of the American Statistical Association 83(403), 596-610.
Cleveland, W. S. and Grosse, E. H. (1991). Computational methods for local regression. Statistics and Computing 1, 47–62.
Cleveland, W.S., Grosse, E., and Shyu, W.M. (1992). Local regression models. Chapter 8 In: Statistical Models in S, edited by J.M. Chambers and T.J. Hastie, Chapman & Hall/CRC, Boca Raton.
Cleveland W.S., and Loader C. (1996). Smoothing by local regression: principles and methods. In: Härdle W., Schimek M.G. (eds) Statistical Theory and Computational Aspects of Smoothing. Physica-Verlag, Heidelberg.
Loader, C. (1999). Local Regression and Likelihood. Springer, New York.