Can somebody explain to me the difference between linear model and loess model in statistics? I need to explain this to non-math people.


A VERY non-technical answer

A simple linear model fits a straight line through a set of points. The line is the best possible straight line (at least, for one sensible definition of best)

A loess model fits a complicated curve through a set of points. In some ways, it can be thought of as a complicated moving average. It is the best possible curve (at least, for one sensible definition of best)

  • $\begingroup$ +1 Out of curiosity, just what is the "one sensible definition of best"? $\endgroup$ – whuber Mar 22 '12 at 20:58
  • $\begingroup$ It is the sum of squared errors, at least for simple regression. For loess, it's a weighted sum of squared errors. $\endgroup$ – Peter Flom - Reinstate Monica Mar 22 '12 at 21:02
  • $\begingroup$ Note that a linear model can be used to fit curves by including polynomial or spline terms. The model is linear in the coefficients, not necessarily limited to a straight line relationship. $\endgroup$ – Greg Snow Mar 22 '12 at 21:30
  • $\begingroup$ I don't think Loess optimizes any criterion of "best," except when the range is 0, in which case the result is trivial: the Loess fit passes through all the points! $\endgroup$ – whuber Mar 23 '12 at 2:20

The loess.demo function in the TeachingDemos package for R will interactively demonstrate the ideas behind a loess fit. It will plot a set of data and the loess fit, then when you click on a point it will show the window used to fit at that point, the relative weights of the points within the window, and the "linear model" fit to that weighted data. Clicking on additional points will then update the display to show the general concept of the loess fit.

This may help explain what loess does and may help in an explanation of the difference.

  • $\begingroup$ how do you run loess.demo in R? $\endgroup$ – mike Mar 23 '12 at 13:17
  • $\begingroup$ @mike, first install R (if you have not already), run R and install the TeachingDemos package (exactly how depends on your system), load the package with library(TeachingDemos) then type ?loess.demo to bring up the help page to see how to run it, you can scroll to the bottom where the example are and copy and paste that code to R's command line to see the examples, then run with your own data to further explore. $\endgroup$ – Greg Snow Mar 23 '12 at 17:15

Here is a simple but detailed response.

A linear model fits a relationship through all of the data points. This model can be first order (another meaning of "linear") or polynomial to account for curvature, or with splines to account for different regions having a different governing model.

A LOESS fit is a locally moving weighted regression based on the original data points. What's that mean?

A LOESS fit inputs the original X and Y values, plus a set of output X values for which to compute new Y values (usually the same X values are used for both, but often fewer X values are used for fitted X-Y pairs because of the increased computation required).

For each output X value, a portion of the input data is used to compute a fit. The portion of the data, generally 25% to 100% but typically 33% or 50%, is local, meaning it is that portion of the original data closest to each particular output X value. It is a moving fit, because each output X value requires a different subset of the original data, with different weights (see next paragraph).

This subset of input data points is used to perform a weighted regression, with points closest to the output X value given greater weight. This regression is usually first order; second order or higher is possible, but require greater computation power. The Y value of this weighted regression calculated at the output X is used as the model's Y value for this X value.

The regression is recomputed at each output X value to produce a full set of output Y values.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.