I am trying to understand LOESS regression. I've read the Wikipedia and Wolfram articles on it, and the R help for the loess() function.
I think I understand now how the local polynomials are derived, each by a weighted regression over a subset of the data points that are all sufficiently close to the point being estimated.
What I have been unable to find any explanation of is how a single smooth (here presumably meaning once or twice differentiable) curve across the whole explanatory variable domain is obtained from that collection of polynomials.
I have entertained various possibilities, such as:
- the domain is chopped into a number of separated intervals, each containing a single data point, with the relevant local polynomial applying in each such interval and an interpolation method being used to bridge the space between one such interval and the next; or
- each of the polynomials is applied across the entire domain and the final curve is an average of these; or
- the local polynomials will naturally merge smoothly into one another in sequence, as a consequence of some property of the estimation procedure that was not highlighted in the above reading
I would be grateful if somebody could explain, or point me to a source that contains an explanation, how a single smooth curve is obtained from the set of multiple local polynomials.
Thank you
Andrew
EDIT: I investigated the suggestion that the curve is piecewise-linear through the fitted points, using the R code below. The differences between the piecewise-linear curve and the loess curve points are of the order of 10^-4 times the fitted values, which is small, but not small enough to be explained as rounding error. I attach a plot of the differences. There's something interesting going on here about how the local polynomials are joined, but I can't find any explanation of what it is.
delta<-0.01
func<-function(x) sin((7/8)*(x*2*pi/delta)) # chose a wiggly function to push the boundaries of the regression
x<-delta*seq(-500,500,1)
y<-func(x)
data<-data.frame(x=x,y=y)
data<-data[order(data$x),]
loess.model<-loess(y~x,data=data,span=0.1,degree=2) # fit the loess model
data$fitted<-loess.model$fitted
x.with.finer.grid<- min(x)+seq(0,10*delta,delta/10)
# make vector of loess predictions at and between fitted points
pred.loess<-predict(loess.model, newdata=x.with.finer.grid)
dev.off()
plot(x=x.with.finer.grid,y=pred.loess,tck=1,xaxp=c(min(x),min(x)+10*delta,10))
# create vector of piecewise linear line connecting fitted points
pred.linear<-approx(x=data$x[1:11],y=data$fitted[1:11],xout=x.with.finer.grid)
dev.off()
plot(x=x.with.finer.grid,y=pred.linear$y,tck=1,xaxp=c(min(x),min(x)+10*delta,10))
dev.off()
diff<-pred.linear$y-pred.loess
plot(diff)
lines(diff)
quickview<-data.frame(loess=pred.loess,linear=pred.linear$y,diff=diff)
quickview