The most common distinction I've seen made between interpolation and extrapolation is that interpolation is within the range of the data, whereas extrapolation is outside the range of the data. This makes sense in one dimension, but in higher dimensions I don't think it works as well. Consider the the following parameter space:
Let's call the function I want to estimate $f(x, y)$. If I want to estimate the value of $f$ at the red dot in the parameter space, is this interpolation or extrapolation?
Using the "range of the data" definition it's interpolation. But the sort of problem described in this answer can still occur here. Data does not exist on all sides of the point in some sense.
Are there more precise definitions of interpolation and extrapolation that don't have this issue?