11
$\begingroup$

What are your thoughts about using regression to project outside of the data range? If we are sure that it follows a linear or power model shape, couldn't the model be useful beyond the data range? For example I have volume driven by price. We should be able to project for prices outside the data range I believe. Your thoughts?

VOL     PRICE
3044    4.97
2549    4.97
3131    4.98
2708    4.98
2860    4.98
2907    4.98
3107    4.98
3194    4.98
2753    4.98
3228    4.98
3019    4.98
3077    4.99
2597    4.99
2706    4.99
3000    4.99
3022    4.99
3084    4.99
3973    4.99
3675    4.99
3065    4.99
3407    4.99
2359    4.99
2802    4.99
2589    4.99
2476    4.99
2387    5
3265    5
2039    5.14
1842    5.15
2660    5.37
1796    5.46
1734    5.46
1881    5.46
2204    5.58
1477    5.77
1620    5.84
1909    5.87
1744    5.87
1247    5.87
1848    5.88
1641    5.88
1758    5.88
1718    5.88
1656    5.88
1822    5.88
1556    5.89
1643    5.9
1850    5.91
1901    5.91
1837    5.91
1773    5.92
1729    5.92
Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Depends on how well your assumptions hold outside the data range. Predicting $y$'s for unobserved $x$'s is the whole reason you do regression in the first place. $\endgroup$
    – Ben
    Oct 15, 2015 at 15:49
  • 3
    $\begingroup$ Even when you're really, really, sure the linear relationship continues beyond the range of the predictors $x_1, \ldots, x_n$ in your sample of $n$ observations, there's a term in the variance of the predicted response for a new observation $x^*$ - viz $\frac{(x^* - \bar{x})^2}{\sum_i^n(x_i - \bar{x})^2}$ - that should worry you. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Oct 15, 2015 at 15:57
  • $\begingroup$ Ben I tend to agree, isn't predicting y's for unobserved X's the whole point? Else why even do a regression? Maybe setting a limit on how far I allow myself to venture away from the observed data range could be responsible. Surely 10% would be safe.. no? $\endgroup$
    – Johnson Jason
    Oct 15, 2015 at 16:02
  • $\begingroup$ My favourite yardstick on this topic is bmj.com/content/317/7155/409. $\endgroup$
    – Carlo Lazzaro
    Oct 21, 2015 at 4:31
  • $\begingroup$ @Ben,@Johnson - Perhaps a quibble. There is another use for regression. It can be used to explain rather than predict. I think, particularly in the social sciences this is a major use of regression. I've read lots of arguments like We think that (variables) A effect outcome B, we run a regression, find that the 95% confidence interval of the coefficient(s) of A don't contain 0 and we conclude that there is a relationship of the form A causes B. Incidentally, this is not something I ever do ! $\endgroup$
    – meh
    Oct 25, 2015 at 17:14

3 Answers 3

Reset to default
13
$\begingroup$

Almost all answers and comments warn against the dangers of extrapolation. I would like to offer a more formal way of seeing whether prediction is prudent. The method is based on the projection matrix on the space spanned by the columns of $\mathbf{X}$ which we assume full rank, i.e. we assume the column space is p-dimensional. As you might remember,

$$\mathbf{H}=\mathbf{X}\left(\mathbf{X}^{T}\mathbf{X} \right)^{-1} \mathbf{X}$$

It can be shown that the diagonal elements of $\mathbf{H}$ satisfy $0<\mathbf{H}_{ii}<1,\ i=1,\ldots,n$, this is a consequence of idempotence by the way, and they can be interpreted as distances from the centroid of the predictor space. This is true because there is a one-to-one correspondence between the leverages $\mathbf{H}_{ii}$ and the squared Mahalanobis distances. A way to spot hidden extrapolations would then be to see how far the new obsevation lies from the centroid, right? This can be done by computing the new diagonal element. Recalling some basic rules of matrix multiplication, we have

$$\mathbf{H}_{new,new} = \mathbf{x}_{new}^{T} \left(\mathbf{X}^{T}\mathbf{X} \right)^{-1} \mathbf{x}_{new} $$

If $\mathbf{H}_{new,new}$ is much larger than the rest of the diagonal elements, then this tells you that your new observation lies quite far from the centroid and prediction is probably a risky move. It takes some judgement to decide how large is too large so of course the technique is not foolproof. Its beauty nevertheless is that it works in all dimensions, when you cannot look at a simple scatter plot that is.

I am not sure which software you are using but almost all of them will return the hat matrix with the right command. So I suggest you take a look before making up your mind.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Well done JohnK, this is very helpful. FYI I am using Excel regression. $\endgroup$
    – Johnson Jason
    Oct 15, 2015 at 18:59
10
$\begingroup$

The prediction error increases quadratically with the distance from the mean. The regression equation and results allow you to gauge the size of the error over the observed range of data, and the model is only adequate over that same range.

Outside of that range a lot of things can happen. First, the prediction gets worse and worse due to the increase of the prediction error.

Second, the model may break down completely. The easiest way to see that is to try to project a model relating price to time: You can't make predictions for negative time.

Third, the linear relationship may be inadequate. In your example, there almost certainly are economies of scale, which would become very noticeable if you try to predict far outside of the range of observed values.

A humorous example of this same effect appears in one of the works of Mark Twain, where he attempts to model the length of the Mississippi river over time --- it it/was quite windy and shortens/ed each year due to erosion of some of the bends as well as man-made shortcuts --- and "predicts" that in so many years the distance between Cairo, Illinois, and New Orleans will have shrunk to about a mile and three quarters).

Finally, note that the range of observed values can be quite complicated if you have more than one predictor variable. (Due to correlations between the predictors you often cannot just take the box defined by the maxima and minima in each predictor.)

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ (+1) Though to say the model is adequate only over the range of the observed data is a bit strong - it's that the problems you describe become more & more concerning the further away you get from it. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Oct 15, 2015 at 16:02
  • $\begingroup$ So is there any work around how far is a safe distance to venture away from the observed data range? Less than 1 standard deviation alright? $\endgroup$
    – Johnson Jason
    Oct 15, 2015 at 16:04
  • 1
    $\begingroup$ @Scortchi. Point taken. In most situations the degradation of the model is gradual. However, occasionally there are hard boundaries, and trying to go beyond those is going to cause grief. $\endgroup$
    – user3697176
    Oct 15, 2015 at 16:17
  • 1
    $\begingroup$ @JohnsonJason: There's no sense in looking for a rule of thumb. You can easily calculate prediction intervals, assuming your model can be extrapolated; the degree to which you can trust extrapolation depends on subject matter knowledge: what's acceptable varies from case to case. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Oct 15, 2015 at 16:17
  • 1
    $\begingroup$ Excellent points (+1). But there is no logical problem in predicting price for negative time. The real problem is if you predict negative price for some given time (usually in the past, in practice). Often that means the model is qualitatively wrong as much as that extrapolation is stretching a line (or curve) too far. A logarithmic link function for example always implies positive predictions. $\endgroup$
    – Nick Cox
    Oct 15, 2015 at 18:30
4
$\begingroup$

You cannot make data driven decisions for areas where you don't have data. End of story. The data can very well support a linear shape for the range of which your data is collected but you do not have data-driven reasons to believe this shape continues to be linear outside your range. It could be any shape under the sun!

You may assume the linear shape continues outside your data range but this is a subjective assumption not supported by the data you've collected. I would suggest consulted a subject matter expert to see, based on their subject matter expertise, how safe this assumption is.

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ So what's the point really of doing regression if we can't predict Y's for unobserved X's $\endgroup$
    – Johnson Jason
    Oct 15, 2015 at 16:04
  • 3
    $\begingroup$ I think the point is that you can still predict inside the range, it's just not advisable to predict outside the range. Presumably most new data points would be inside the range, so the model would remain useful the vast majority of the time $\endgroup$
    – Ryan Zotti
    Oct 15, 2015 at 16:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.