1
$\begingroup$

I am trying to understand the difference between these two stated population linear regression models. My professor's notes state that, if we have predictors $X_1, \dots, X_p$, then the population linear regression model is $Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \dots \beta_p X_p + \epsilon$. But I have also seen the population linear regression model stated as $Y_i = \beta_0 + \beta_1 X_i + \epsilon_i$. What is the difference between these two, and are they both actually the population linear regression model?


EDIT

If you Google "linear regression population model," you'll find these images:

enter image description here

enter image description here

enter image description here

$\endgroup$
5
  • $\begingroup$ I don't think either of those make sense for the term, but especially that second equation makes absolutely no sense $\endgroup$
    – bdeonovic
    Nov 3, 2021 at 16:07
  • $\begingroup$ Ah I think I understand the confusion I will post in an answer in a few min. $\endgroup$
    – bdeonovic
    Nov 3, 2021 at 16:09
  • $\begingroup$ @bdeonovic If you Google "linear regression population model," you'll find these images (see my edit). $\endgroup$ Nov 3, 2021 at 16:10
  • $\begingroup$ Surprisingly people on the internet can be wrong, or glossing over details $\endgroup$
    – bdeonovic
    Nov 3, 2021 at 16:20
  • $\begingroup$ The slides you posted describe "simple" linear regression (ie linear regression with just one covariate) so they have dropped the index for covariate number $\endgroup$
    – bdeonovic
    Nov 3, 2021 at 16:21

1 Answer 1

1
$\begingroup$

The linear regression model is

$$y_i = \beta_0 + \beta_1 x_{1i} + \cdots + \beta_p x_{pi} + \epsilon_i$$

for $i=1,\ldots,n$ and $\epsilon_i \sim N(0,\sigma^2)$ and independent

Here $y_i$ is the outcome (or dependent variable) for unit $i$ (e.g. person $i$). Sometimes this indexed is dropped and simply stated as

$$y = \beta_0 + \beta_1 x_{1} + \cdots + \beta_p x_{p} + \epsilon$$

"simple" linear regression is linear regression with just one covariate in which case you have $$ y_i = \beta_0 + \beta_1 x_i + \epsilon_i $$ Note we can drop one of the indexes on the covariate $x$ to indicate which of our covariates it is, since there is only 1.

The "population" linear regression isn't a well defined term. Probably most people would assume this means the population mean in the above linear regression which would simply be $$\mu = \beta_0 + \beta_1 x_{1} + \cdots + \beta_p x_{p}$$

Additionally, linear regression can be expressed more compactly in matrix notation as

$$ \mathbf{y} = \mathbf{X}\mathbf{\beta} + \mathbf{\epsilon}$$

Where $\mathbf{y}$ is the $n\times 1$ vector $\mathbf{y}=(y_1,\ldots,y_n)$, $\mathbf{X}$ is the $n\times p$ matrix

$$\begin{pmatrix} x_{11} & \cdots & x_{1p}\\ & \ddots & \\ x_{n1} & \cdots & x_{np} \end{pmatrix}$$ and $\epsilon$ is the $n\times 1$ vector $\epsilon = (\epsilon_1,\ldots,\epsilon_n)$

$\endgroup$
3
  • $\begingroup$ Thanks for the answer. When my professor says "population" linear regression model, I think they are contrasting it with the estimator model $\hat{y} = \hat{\beta_0} + \hat{\beta_1} x_1 + \dots + \hat{\beta_p} x_p$, since this is covered in the same notes. So the use of the word "population" here would be to differentiate the model from the "realised"/"estimator" model. Makes sense, right? $\endgroup$ Nov 3, 2021 at 16:27
  • $\begingroup$ that could be what they meant, but that is not typically how the term "population" is used. It typically refers to the mean $\mu = \beta_0 + \beta_1x_1 + \cdots + \beta_p x_p$. $\epsilon$ refers to the individual variation away from this mean. That is why this value $\mu$ is typically referred to as the "population" mean $\endgroup$
    – bdeonovic
    Nov 3, 2021 at 16:42
  • $\begingroup$ Ok, thank you for taking the time to clarify this. $\endgroup$ Nov 3, 2021 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.