I am trying to understand LOESS regression. I've read the Wikipedia and Wolfram articles on it, and the R help for the loess() function.
I think I understand now how the local polynomials are derived, each by a weighted regression over a subset of the data points that are all sufficiently close to the point being estimated.
What I have been unable to find any explanation of is how a single smooth (here presumably meaning once or twice differentiable) curve across the whole explanatory variable domain is obtained from that collection of polynomials.
I have entertained various possibilities, such as:
- the domain is chopped into a number of separated intervals, each containing a single data point, with the relevant local polynomial applying in each such interval and an interpolation method being used to bridge the space between one such interval and the next; or
- each of the polynomials is applied across the entire domain and the final curve is an average of these; or
- the local polynomials will naturally merge smoothly into one another in sequence, as a consequence of some property of the estimation procedure that was not highlighted in the above reading
I would be grateful if somebody could explain, or point me to a source that contains an explanation, how a single smooth curve is obtained from the set of multiple local polynomials.
EDIT: I investigated the suggestion that the curve is piecewise-linear through the fitted points, using the R code below. The differences between the piecewise-linear curve and the loess curve points are of the order of 10^-4 times the fitted values, which is small, but not small enough to be explained as rounding error. I attach a plot of the differences. There's something interesting going on here about how the local polynomials are joined, but I can't find any explanation of what it is.
delta<-0.01 func<-function(x) sin((7/8)*(x*2*pi/delta)) # chose a wiggly function to push the boundaries of the regression x<-delta*seq(-500,500,1) y<-func(x) data<-data.frame(x=x,y=y) data<-data[order(data$x),] loess.model<-loess(y~x,data=data,span=0.1,degree=2) # fit the loess model data$fitted<-loess.model$fitted x.with.finer.grid<- min(x)+seq(0,10*delta,delta/10) # make vector of loess predictions at and between fitted points pred.loess<-predict(loess.model, newdata=x.with.finer.grid) dev.off() plot(x=x.with.finer.grid,y=pred.loess,tck=1,xaxp=c(min(x),min(x)+10*delta,10)) # create vector of piecewise linear line connecting fitted points pred.linear<-approx(x=data$x[1:11],y=data$fitted[1:11],xout=x.with.finer.grid) dev.off() plot(x=x.with.finer.grid,y=pred.linear$y,tck=1,xaxp=c(min(x),min(x)+10*delta,10)) dev.off() diff<-pred.linear$y-pred.loess plot(diff) lines(diff) quickview<-data.frame(loess=pred.loess,linear=pred.linear$y,diff=diff) quickview