How to make predictions with non-parametric regression? Let's say I have a dataset to which I have estimated a relationship using non-parametric regression, specifically Kernel (obviously in this hypothetical example it's probably overfit slightly). The dataset is from x = 0 to x = 4. How would I go about extrapolating, and finding the y-value for say x = 4.2?
Would I simply extend the last connecting line of the regression, as depicted below? Or is this incorrect?
The reason I want to do this, is because I want to calculate LOOCV - which involves omitting each data point in turn, and I'm unclear on what to do when I omit the first last or data point in a range.

(Please be aware this is just randomly generated data, for illustrative purposes).
Thank you
Edit 1: Emphasis on reason for asking.
 A: Typically I have seen that done just by extending the last line forward, although this obviously can be quite dangerous because of the nature of that local fitting and extreme values near the end points.
There may be some advanced techniques for it or you could take some notes from time-series literature and 'dampen' the last line or combine it with a global measure for the trend.  So you use that last line averaged with the overall slope of a simple regression or something to try and keep your last local measure in line.  These are all just off the cuff suggestions but no matter what you do extending it outside of your fitted x-value range is possible but dangerous.
A: You can't. Machine learning models, broadly speaking, learn to interpolate between the data points. Here you are trying to extrapolate, extrapolation is hard and can easily go wrong. If you used a simple model, say linear regression, then it is easy to extrapolate and fairly easy to assess how bad the result could go. If you are using a more complicated model, say a polynomial regression, then as in the extract of the xkcd comic posted below, it can go arbitrarily bad.

Models like random forest or kernel regression can only interpolate between the data points. To extrapolate beyound the data, you could do something like using the last seen slope (as you did on the plot attached to the question), or just use the prediction for the last known point (x=4) as predictions for everything above. Such simple methods for extrapolating are commonly used for time-series and may be preferred if you don't have much data. Both solutions are rather arbitrary and there is no good answer.
