0
$\begingroup$

I am doing an online course and came across this line- "There is a population regression line that joins the mean of the dependent variables". I was stumped by this because if I took the mean of all the observed values (the $y_is$), which are the dependent variables, it would give me a single value. There are no coordinates associated with this value and I am not sure what the said line is 'joining'.

I did some research and found that whenever we perform linear regression, we always do it on a sample of the population, since we usually do not have access to the data of the whole population. Therefore we can only estimate the parameters $\beta_0$ and $\beta_1$ (I am assuming a univariate model). Suppose I did have access to the population data. The problem I run into is that there can be multiple $y_i$ values associated with each value of the predictor $x_i$. As an example, suppose I am drawing up a model of weekly expenditure (dependent variable) against annual income (independent variable. For an annual income of say 30,000 there could be people who spend 50, 60, 65 etc. per week for that particular income bracket.

My question: Even if we do have access to the population data, can we construct a population regression model that would give me the true values of $\beta_0$ and $\beta_1$? My guess was that we would take the mean of all the observed $y_is$ at each value of $x_i$ to get $\overline{y_i}$and then use those means to get the model. Is this what the aforementioned statement means when it says that the population regression line "joins the mean of the dependent variables"?

Edit: enter image description here

I am not sure if this image represents the population regression line but this is what a Google search churned out. Again, we can see that for each value of the duration, there are multiple values of the interval.

$\endgroup$
5
  • $\begingroup$ This will help clarify what you want to ask: what do you mean by all observed $y_is$ at each value of $x$ when we're working at the population level? $\endgroup$
    – Dave
    Jul 2, 2020 at 19:51
  • $\begingroup$ Expanding on the example in my question, suppose I take a look at all the people whose annual income is 30,000. Now each of these might spend a different amount each week; some may spend 50, some 60, 65 etc. So at each income level, there is not one particular value that each person is likely to spend, it's a distribution of several values. Similar would be the case for people with annual income of 40,000. This is what I mean by multiple $y_is$ for each $x_i$ at the population level. If I were to take a sample from this population, I would only have one $y_i$ for each $x_i$. $\endgroup$ Jul 2, 2020 at 19:59
  • $\begingroup$ By definition, the "population regression line" is actually a curve whose points are given by all the $(x_i, \bar y_i).$ "The true values of $\beta_0$ and $\beta_1$" has a doubtful meaning unless this curve is part of a line. $\endgroup$
    – whuber
    Jul 2, 2020 at 20:07
  • $\begingroup$ @whuber I don't quite get you. Why would I have a curve if I am fitting a model of the form $y = \beta_0 + \beta_1x$? I have added an image, if that helps convey what I am trying to say. The meaning of the statement at the beginning still eludes me though. $\endgroup$ Jul 3, 2020 at 5:12
  • 1
    $\begingroup$ I think @whuber is making the point that the true relationship need not be a line. You are choosing to fit a line (which is up to you); the true relationship is up to Nature. Sometimes it matters whether we're talking about a decision to fit a line or an assumption that the truth is a line, sometimes it doesn't. Here, it does. $\endgroup$ Jul 3, 2020 at 6:39

1 Answer 1

3
$\begingroup$

This isn't really about specific fixed populations, it's about models for the data-generating process.

The model we have is that there are $X$s. For each possible value $x$ there are infinitely many possible $y$ values, forming a probability distribution. We only observe a few values $x_i$ and only one $y_i$ for each $x_i$.

One possible model is that the mean of $Y$ for each value of $X$ is described by a straight line: $$E[Y|X=x]=\beta_0+\beta_1x.$$ Given this model it makes sense to talk about the true line, which is the one with intercept $\beta_0$ and slope $\beta_1$. The true line goes through $(x, E[Y|X=x])$ for every $x$. The model also gives enough mathematical structure to talk about how the least-squares estimates $\hat\beta_0$ and $\hat\beta_1$ relate to the true $\beta_0$ and $\beta_1$. It also lets us talk about values of $x$ where we haven't observed any $Y$.

Even if you have data from a large (finite) population and the model is true, the averages of $y$ for each distinct value of $x$ will typically not trace out the line. You typically won't have enough observations with a specific value of $x$ to guarantee that the average of their $y$s is close to $E[Y|X=x]$. To say the population line "joins the mean of the dependent variables" really requires a probability model for a potentially infinite collection of observations.

As a separate issue, because it came up in comments: this model might not be true, even approximately. In general, the curve traced out by $E[Y|X=x]$ is not a straight line, and it need not be close to a straight line. If it isn't, you need an expanded model, one that allows for varying shapes of the curve. In that setting, if you wanted to be careful about inference, you could either

  • try to estimate $E[Y|X=x]$, by estimating the whole shape of the curve
  • decide you wanted the best-fitting straight line $\hat\beta_0+\hat\beta_1 x$, and then consider what the properties of this line would be if the truth is not a straight line.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.