Suppose that we have two points (the following figure: black circles) and we want to find a value for a third point between them (cross). Indeed we are going to estimate it based on our experimental results, the black points. The simplest case is to draw a line and then find the value (i.e., linear interpolation). If we had supporting points e.g., as brown points in both sides we prefer to get benefit from them and fit a non-linear curve (green curve).

The question is that what is the statistical reasoning to mark the red cross as the solution? Why other crosses (e.g., yellow ones) are not answers where they could be? What kind of inference or (?) pushes us to accept the red one?

I will develop my original question based on the answers got for this very simple question.

enter image description here

  • 7
    $\begingroup$ This is a very well posed and interesting question. You might want to distinguish between time series interpolation and other forms of interpolation (such as splining or spatial interpolation), due to the inherent directionality of time series. $\endgroup$ – whuber Oct 4 '11 at 18:22
  • 1
    $\begingroup$ My appreciation goes to this very motivational comment. $\endgroup$ – Developer Oct 8 '11 at 13:49
  • $\begingroup$ See also How Does Kriging Interpolation work?. $\endgroup$ – Scortchi Aug 4 '15 at 12:48

Any form of function fitting, even nonparametric ones (that typically make assumptions on the smoothness of the curve involved), involves assumptions, and thus a leap of faith.

The ancient solution of linear interpolation is one that 'just works' when the data you have is fine-grained 'enough' (if you look at a circle close enough, it looks flat as well - just ask Columbus), and was feasible even before the computer age (which is not the case for many modern day splines solutions). It makes sense to assume the belief that the function will 'continue in the same (i.e. linear) matter' between the two points, but there is no a priori reason for this (barring knowledge about the concepts at hand).

It becomes quickly clear when you have three (or more) noncolinear points (like when you add the brown points above), that linear interpolation between each of them will soon involve sharp corners in each of those, which is typically unwanted. That is where the other options jump in.

However, without further domain knowledge, there is no way to state with certainty that one solution is better than the other (for this, you would have to know what the value of the other points is, defeating the purpose of fitting the function in the first place).

On the bright side, and maybe more relevant to your question, under 'regularity conditions' (read: assumptions: if we know that the function is e.g. smooth), both linear interpolation and the other popular solutions can be proven to be 'reasonable' approximations. Still: it requires assumptions, and for these, we typically do not have statistics.

  • $\begingroup$ This is a good answer and is my candidate to be marked as the answer. I understood that there is no statistical justification for such a common choice, right? $\endgroup$ – Developer Oct 4 '11 at 19:02
  • $\begingroup$ Indeed I believe there isn't one, no. $\endgroup$ – Nick Sabbe Oct 4 '11 at 21:05
  • 2
    $\begingroup$ Some literature (involving competitions to interpolate samples of well-known datasets) partially validates this reply, but not entirely. One can learn much about the spatial correlation of the data through statistical analysis of the data alone, without any "regularity conditions." What is needed are a model of the data as a sample of one realization of a stochastic process along with (1) an ergodic hypothesis and (in most cases) (2) some kind of stationarity assumption. In this framework interpolation becomes prediction of an expectation, but even nondifferentiable curves are allowed. $\endgroup$ – whuber Oct 4 '11 at 21:33
  • 1
    $\begingroup$ @whuber: I'm way out of my comfort zone here, but everything after "regularity conditions" in your comment reads like a fairly solid amount of assumptions (stationarity would likely amount to a regularity condition, no?). Actually, I think it's going to depend on whether your sample size is large with respect to the irregularities in the functional form... Can you give a reference of a paper or the likes where this isn't the case? $\endgroup$ – Nick Sabbe Oct 4 '11 at 22:20
  • 2
    $\begingroup$ You can't do anything without assumptions, Nick! But regularity (such as smoothness of the function) is not necessary: it can be deduced from the data, at least on the scale at which the function is sampled. (Stationarity is a much milder assumption than smoothness.) You are correct that largish samples are needed, but much can be learned in 2D even with 30-50 well chosen sample locations. The literature is large; for instance, most of the issues of Mathematical Geology are devoted to this. For a rigorous introduction, see Cressie's Spatial Statistics. $\endgroup$ – whuber Oct 4 '11 at 23:06

You can work out the linear equation for the line of best fit (eg. y = 0.4554x + 0.7525 ) however this would only work if there was a labeled axis. However this would not give you the exact answer only the best fitting one in relation the other points.

  • $\begingroup$ But regression isn't interpolation. $\endgroup$ – Scortchi Aug 4 '15 at 9:31
  • 1
    $\begingroup$ @Scortchi I believe regression can be understood as interpolation. However, proposing regression as a solution does not answer the question, which asks us to explain why any kind of interpolation is justifiable (and implicitly invites us to describe the assumptions needed to justify it). $\endgroup$ – whuber Aug 4 '15 at 13:11
  • $\begingroup$ @whuber: Thanks. I was thinking of interpolation, prototypically at least, as join-the-dots - stats.stackexchange.com/a/33662/17230. $\endgroup$ – Scortchi Aug 4 '15 at 14:04
  • $\begingroup$ @Scortchi That thread addresses primarily the mathematical concept of interpolation in a table. In a comment to its question I pointed out the conventional statistical understanding of interpolation, which is subtly different. Regression works in both worlds: a regression function can serve as a mathematical interpolator (for a well-defined function that is sampled in a table) as well as a statistical interpolator (by means of statistical predictions of values of a stochastic process conditional on a finite number of values derived from that process). $\endgroup$ – whuber Aug 4 '15 at 14:13
  • 1
    $\begingroup$ @Cagdas The only way to perfectly reconstruct a function from finite data is to supply enough restrictions on the function that there is only one candidate for it conditional on the data! In particular, given the number of data points $n$ and given the function's supports (but independent of its values at those supports), the set of possible functions must be a finite-dimensional manifold of dimension at most $n$. $\endgroup$ – whuber Aug 4 '15 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.