Question about the Multiple Linear Regression: why and how does it work?

Question

I know this question is quite simple and maybe quite naive as well, but I would like to get some help. The general linear model can be expressed as \begin{align*} \textbf{Y} = \textbf{X}\beta + \epsilon \end{align*}

where $Y\sim\mathcal{N}(\textbf{X}\beta,\sigma^{2}\textbf{I})$ represents the random component, $\textbf{X}\beta$ represents the systematic component and the link function is given by the identity $g(\mu) = \mu = \textbf{X}\beta$.

My question is: why do we assume the response variable $\textbf{Y} = (Y_{1},Y_{2},\ldots,Y_{n})$ equals the mean $\mu = \textbf{X}\beta$ plus an error $\epsilon$, which is normally distributed? Moreover, how do we interpret the mean of each component $Y_{i}$? Since each $Y_{i}$ is an observation from the random variable whose distribution describes the data, why should them have different means? Does each $Y_{i}$ represent a "person" from the target population?

Here it is an example. Consider that $\mu_{i} = \beta_{0} + \beta_{1}x_{i1} + \beta_{2}x_{i2}$, where $\mu_{i}$ indicates the average income from the population that lives in the city $i$, $1\leq i\leq 3$, and the $x_{ij}$ represent some features which influence its value. Then, most probably, we will obtain different values for the means $\mu_{1}$, $\mu_{2}$ and $\mu_{3}$. Why does it sound reasonable to state that $Y_{i} = \mu_{i} + \epsilon_{i}$, where $\epsilon$ is normally distributed and $Y_{i}$ corresponds to the income from some habitant which lives in city $i$?

Any help is appreciated. Thanks in advance!

Answer 1

To answer your questions in turn:

why do we assume the response variable $\textbf{Y} = (Y_{1},Y_{2},\ldots,Y_{n})$ equals the mean $\mu = \textbf{X}\beta$ plus an error $\epsilon$, which is normally distributed?

This is equivalent to saying that $Y\sim\mathcal{N}(\textbf{X}\beta,\sigma^{2}\textbf{I})$, that is, that $Y$ is a random variable with normal distribution, conditional on the covariates $\textbf{X}$, specifically, the mean of $Y$ is a linear function of the covariates.

Moreover, how do we interpret the mean of each component $Y_{i}$?

As written, $Y_i$ is a random variable, so the mean is the expected value $\mathbb{E}(Y | \textbf{x}_i)$. Your regression tells you something about the distribution of $Y$ corresponding to a population taking a specific value for the covariates. The mean is a convenient way to summarise this distribution.

Does each $Y_{i}$ represent a "person" from the target population?

$Y_i$ is not a "person". $Y_i$ is a random variable that models some quantity of interest with regard to the target population. Using the classic example of regressing weight versus height, a population we might be interested in is all people with height of 180 cm. Then, while modelling, we assume that the weight of people with height of 180 cm is normally distributed about some mean. In this example, at a personal level, $y_i$, a random variate of $Y_i$, is the weight observation of a person with height of 180 cm.

Why does it sound reasonable to state that $Y_{i} = \mu_{i} + \epsilon_{i}$, where $\epsilon$ is normally distributed?

Maybe these questions might help?

Is there an explanation for why there are so many natural phenomena that follow normal distribution?

Why do we assume that the error is normally distributed?