3
$\begingroup$

Suppose we have a dataset where the indepedent variable $x$ is the work experience in years of an employee and $y$ is his salary in dollars. Such a dataset could consist of the following elements

$$(x_i , y_i) = \{(1, 30000), (3, 40000), (5, 50000), (7, 60000), (9, 70000)\}$$

The linear regression model will be $y=\theta_0+\theta_1 x$. We can estimate the parameter vector $\theta=[\theta_0, \theta_1]$.

Now, what I've been taught is that the intercept $\theta_0$ is the expected salary of an employee with $0$ years of experience. This is obvious because we just say $x=0$ in the regression equation and we receive the value.

However, what we are technically doing is that we are estimating a value of $y$ given a value of $x$ that does not belong in our dataset. The value $x=0$ is outside of the range of values that $x$ takes in our known data points.

I know that when we do this, it wont always result in a correct conclusion because we are assuming that the relationship of $x,y$ is still linear outside of this range.

So, is it actually correct/safe to just plug in the value $x=0$ and say that it gives us the expected value of $y$ when $x$ takes the value $0$?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Could you explain what you mean by "correct" or "safe"? Obviously, at a mathematical level, there is nothing incorrect about plugging the value $x=0$ into the fitted equation. $\endgroup$
    – whuber ♦
    Commented Oct 13, 2020 at 17:44
  • $\begingroup$ Isnt it "bad habit" to use linear regression model to estimate $y$ for values of $x$ that are not within your known data range? This is what I mean, and its not safe because outside of this range of $x=1,3,5,7,9$ you dont know if the point can still be modeled with a line. $\endgroup$
    – thenac
    Commented Oct 13, 2020 at 18:17
  • 1
    $\begingroup$ Nobody is advocating such extrapolation. What you have been taught is purely mathematics, not statistics or even interpretation. $\endgroup$
    – whuber ♦
    Commented Oct 13, 2020 at 18:47
  • 2
    $\begingroup$ Literally, the intercept is the predicted value when $x=0.$ If you seek a meaningful value, then what you must do is start by re-centering the regressor $x.$ People often center it around its mean, in which case the intercept is the predicted value when $x$ equals its mean (if that makes any sense, which is still not guaranteed). Alternatively, center it around a truly meaningful value. For instance, if an experiment began in 2018 and $x$ is the observation date, then re-expressing it in terms of years since the start of the experiment means the intercept is the predicted value at the start. $\endgroup$
    – whuber ♦
    Commented Oct 13, 2020 at 21:56
  • 1
    $\begingroup$ Ok, thanks a lot for the info!! $\endgroup$
    – thenac
    Commented Oct 13, 2020 at 22:06

1 Answer 1

Reset to default
3
$\begingroup$

What you are talking about here is extrapolation. There is nothing fundamentally, mathematically, incorrect about doing this, but it should be done with care because sometimes linear associations are only linear within a particular range (ie the range of the data you have). If the value you are extrapolating to, is a valid one, then the actual association between the variables may be very different if you had actually obtained data that included that value. It might still be linear, in which case it is likely that the fitted line will have a different slope and intercept, or it might be nonlinear. The other problem, as you have noted, is that sometimes a value of zero does not make sense at all - income, height, weight etc. In this case sometimes an analyst will centre the data about the mean, so that the intercept is then equivalent to the mean.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you for the reply. My point is exactly what you are saying. So my question still remains: Do you perhaps know why, my professor at least, always interprets the intercept by saying that its the expected value of $y$ for $x=0$? Because as you just described, its not always "correct" to do estimations for values of $x$ not within your initial data range. $\endgroup$
    – thenac
    Commented Oct 13, 2020 at 18:46
  • $\begingroup$ You are welcome. Your professor is correct - according to the model the intercept is indeed the expected value of $y$ when $x=0$. Whether this makes sense, or is a sensible thing to do, will depend on the context. In general, extrapolation should be used with extreme caution. But statistically speaking there is nothing wrong with what your professor said: it is description of the model. $\endgroup$
    – Robert Long
    Commented Oct 13, 2020 at 18:50
  • $\begingroup$ Ok! So in general I should avoid interpreting the intercept like this? Is there a better way to do it? (Perhaps the one you noted in your answer regarding centering the data) $\endgroup$
    – thenac
    Commented Oct 13, 2020 at 21:03
  • 2
    $\begingroup$ Personally I would avoid extrapolation unless I thought there was a reason to do it. Yes, centering the data often makes more sense. $\endgroup$
    – Robert Long
    Commented Oct 14, 2020 at 3:23
  • $\begingroup$ Ok, thanks a lot for helping ! :) $\endgroup$
    – thenac
    Commented Oct 14, 2020 at 14:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.