Kernel smoother is a typical non-parametric regression method that can be used well for interpolation. However, it can not predict the data outside the range of training data. Why ? If we have a look at the prediction function :


The prediction function $\hat{Y}(x_0)$ on the provided link will be the average of positively weighted training output. Therefore, $\hat{Y}(x_0)$ can not go over the range of training output. Is there any way to modify kernel smoothing regression for extrapolation ?



You can smooth without strong assumptions on the functional form of the relationship (you need to assume the relationship is smooth and not too rapidly-varying). The data may go up a little, or up a lot, or stay fairly flat or go down, or oscillate, and a smoother can follow it -- smoothing methods achieve this by being adaptive enough to fit the main trend of the data but not so adaptive that they just interpolate noise.

You can't extrapolate without some pretty strong assumptions because there's no data there to adapt to. The values in this unseen set of values might go up a little, or up a lot, or stay fairly flat or go down, or oscillate ...

plot showing three possible smooth futures for a smoothly varying past

... but there's nothing to tell us which it might be (or indeed something else), unless you make suitable assumptions that might give some way of figuring it out and then your out-of-range predictions will rely on those assumptions.

Note that if I had the grey points and any one of the three sets of coloured points a kernel smoother would describe the smooth relationship using all the points very well -- it would fit both halves of the data, and everything would look just great. Now we cut off the coloured part. Which curve might you pick? What about fifty other smooth curves for the second half that I might have used but didn't?

You can modify smoothers in various ways to project, but if you project more than a couple of observations out they'll rely heavily on what, exactly, you do. That really shouldn't be done automatically because you'll need to rely on things like domain knowledge to make sensible choices about constraints (and you'll need some) on what the function can do and how it relates to the trends in the region of x-space that you have.

  • $\begingroup$ Thanks for your reply. Can you suggest how can we encode those smoothness assumption into the model? I am thinking Bayesian inference here, like the assumption can be encoded by a prior distribution but it is still not clear. $\endgroup$ – Lan Trần Thị Mar 8 '17 at 0:42
  • $\begingroup$ Kernel smoothers already have smoothness assumptions built in. Smoothness alone is patently insufficient (as the plot clearly demonstrates), because it only constrains the few values very close to the edge of the data -- beyond that, it could go pretty much anywhere. $\endgroup$ – Glen_b -Reinstate Monica Mar 8 '17 at 0:46
  • $\begingroup$ Is there an interactive app on that shows extrapolations a) gray forward, b) red / green / blue backwards in time ? $\endgroup$ – denis Apr 14 '19 at 8:55
  • $\begingroup$ The red green and blue points are simply three sets of data that are plausible smooth extensions of the grey one. There's no solid basis on which to prefer one to the other purely on "smoothness" grounds, since the possible future data values all "smooth + noise", even across the threshhold $\endgroup$ – Glen_b -Reinstate Monica Apr 14 '19 at 9:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.