0
$\begingroup$

I know this question is quite simple and maybe quite naive as well, but I would like to get some help. The general linear model can be expressed as \begin{align*} \textbf{Y} = \textbf{X}\beta + \epsilon \end{align*}

where $Y\sim\mathcal{N}(\textbf{X}\beta,\sigma^{2}\textbf{I})$ represents the random component, $\textbf{X}\beta$ represents the systematic component and the link function is given by the identity $g(\mu) = \mu = \textbf{X}\beta$.

My question is: why do we assume the response variable $\textbf{Y} = (Y_{1},Y_{2},\ldots,Y_{n})$ equals the mean $\mu = \textbf{X}\beta$ plus an error $\epsilon$, which is normally distributed? Moreover, how do we interpret the mean of each component $Y_{i}$? Since each $Y_{i}$ is an observation from the random variable whose distribution describes the data, why should them have different means? Does each $Y_{i}$ represent a "person" from the target population?

Here it is an example. Consider that $\mu_{i} = \beta_{0} + \beta_{1}x_{i1} + \beta_{2}x_{i2}$, where $\mu_{i}$ indicates the average income from the population that lives in the city $i$, $1\leq i\leq 3$, and the $x_{ij}$ represent some features which influence its value. Then, most probably, we will obtain different values for the means $\mu_{1}$, $\mu_{2}$ and $\mu_{3}$. Why does it sound reasonable to state that $Y_{i} = \mu_{i} + \epsilon_{i}$, where $\epsilon$ is normally distributed and $Y_{i}$ corresponds to the income from some habitant which lives in city $i$?

Any help is appreciated. Thanks in advance!

$\endgroup$
  • $\begingroup$ Not quite sure about some of your terminology here. Usually, $\epsilon_i$ is the random component, and $Y\sim\mathcal{N}(\textbf{X}\beta,\sigma^{2}\textbf{I})$ is the distribution of the random variable $Y$ conditional on the covariates $\textbf{X}$. $\endgroup$ – Alex May 16 at 3:14
  • $\begingroup$ Sorry, it was a typo. I have already fixed it. $\endgroup$ – user1337 May 16 at 3:15
1
$\begingroup$

To answer your questions in turn:

why do we assume the response variable $\textbf{Y} = (Y_{1},Y_{2},\ldots,Y_{n})$ equals the mean $\mu = \textbf{X}\beta$ plus an error $\epsilon$, which is normally distributed?

This is equivalent to saying that $Y\sim\mathcal{N}(\textbf{X}\beta,\sigma^{2}\textbf{I})$, that is, that $Y$ is a random variable with normal distribution, conditional on the covariates $\textbf{X}$, specifically, the mean of $Y$ is a linear function of the covariates.

Moreover, how do we interpret the mean of each component $Y_{i}$?

As written, $Y_i$ is a random variable, so the mean is the expected value $\mathbb{E}(Y | \textbf{x}_i)$. Your regression tells you something about the distribution of $Y$ corresponding to a population taking a specific value for the covariates. The mean is a convenient way to summarise this distribution.

Does each $Y_{i}$ represent a "person" from the target population?

$Y_i$ is not a "person". $Y_i$ is a random variable that models some quantity of interest with regard to the target population. Using the classic example of regressing weight versus height, a population we might be interested in is all people with height of 180 cm. Then, while modelling, we assume that the weight of people with height of 180 cm is normally distributed about some mean. In this example, at a personal level, $y_i$, a random variate of $Y_i$, is the weight observation of a person with height of 180 cm.

Why does it sound reasonable to state that $Y_{i} = \mu_{i} + \epsilon_{i}$, where $\epsilon$ is normally distributed?

Maybe these questions might help?

Is there an explanation for why there are so many natural phenomena that follow normal distribution?

Why do we assume that the error is normally distributed?

$\endgroup$
  • $\begingroup$ In the first place, thanks for the answer. But I still don't understand some things. Do the $Y_{i}$ are related to different samples? Precisely speaking, there are as many samples as the number of $Y_{i}$? Otherwise, if all the outcomes $Y_{i}$ come from the same sample, why do they have different means? $\endgroup$ – user1337 May 16 at 13:25
  • $\begingroup$ each unique set of values of the covariates corresponds to a unique population, which has its own mean. All "people" who share the same covariates come from this population, and the expected value of $Y$ for these people is the mean of the distribution. $\endgroup$ – Alex May 16 at 23:17
  • $\begingroup$ "Precisely speaking, there are as many samples as the number of Yi?" Each observation $(x_i, y_i)$ is a sample but there might not be as many populations of interest as samples. Consider the dataset of people's weights where you have two covariates, sex, which takes values male and female, and height which is fixed at 180 cm. There are only two populations here, every observation that is a woman are treated the same, and every observation that is a man is treated the same. $\endgroup$ – Alex May 16 at 23:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.