# What are the theoretical reasons for why extrapolation “less reliable” than interpolation?

Extrapolation is in general "unreliable". (See "What is wrong with extrapolation?")

But it is also commonly said that extrapolation is "less reliable" than interpolation.

But why should we generally assume that the model is "more reliable" between two known data points than to the right of the right-most data point (or the left of the left-most data point)?

From empirical examples, I can see that indeed, interpolation is often "more reliable" than extrapolation. But is there a more formal, theoretical justification for why this assertion is, in general, true?

Or is it just a purely empirical observation that interpolation tends to be "better"?

• extrapolation “less reliable” than interpolation that is true not in all contexts. It may be true when a specific number, more than one, of known points is demanded to infer the unknown point. In interpolation, you estimate unknown t2 value from t1 and t3 values, both known and both adjacent to t2. Consider now the comparable extrapolation: t1 is known, t3 in not known, extrapolation t2 from t1 can be seen as interpolation from t1 and (unknown) t3 onto t2. Of course, it is less "reliable". – ttnphns Jun 30 '16 at 7:52
• (cont.) Alternatively, the extrapolation can be taken as extrapolation from t1 and t0 (both known) onto t2. But here t0 is "far away", so less "reliable" once again. – ttnphns Jun 30 '16 at 7:52

It is a theoretical result, at least for linear regression. Indeed, if one computes the so-called ''prediction error'' (see this link, slide 11), one can easily see that the further the independent variable $x$ is away from the sample average $\bar{x}$ (and for extrapolation one may be far away), the larger the prediction error. In the link that I referred to one can also see that in a graphical way.
That's the theoretical part. Empirically, we usually assume continuity and smoothness of a response surface, since many processes behave that way. Hence, we assume that the true model comes from a class of well-behaved functions, not crazy, jagged-edged monsters. This is a key regularization step in model fitting. With that assumption, we get the idea that points close in inputs space should be close in their function values, with convergence to $0$ difference as the points approach one another.