Once again, here I am. Given the multiple linear regression model \begin{align*} \textbf{Y} = \textbf{X}\beta + \epsilon \end{align*}

where $\epsilon\sim\mathcal{N}(\textbf{0},\sigma^{2}\textbf{I})$ and $\mu = \textbf{X}\beta$, why do we need to determine the distribution of $\textbf{Y}$? If we apply the least square method to obtain $\hat{\beta}$, we get the explicit relation \begin{align*} Y_{i} = \hat{\beta}_{0} + \hat{\beta}_{1}x_{i1} + \ldots + \hat{\beta}_{p-1}x_{i,p-1} + \epsilon_{i} \end{align*}

from whence we are able to obtain the value of the response variable $Y$ in terms of the explanatory variables. My second question is: how do we interpret each component of $\textbf{Y} = (Y_{1},Y_{2},\ldots,Y_{n})$? Does each $Y_{i}$ represent the outcome from a different sample? Otherwise, if they belong to the same sample, why do they have different means?


Linear regression makes no assumptions on the distribution of the marginal outcome (That is, $\bf{Y}$). However, there is an assumption on the distribution of the elements of $\bf{Y}$.

Each element of $\bf{Y}$ should have a normal distribution. Given the covariates, the distribution should be

$$y \vert \beta,x \sim \mathcal{N}(x^T\beta, \sigma)$$

To your specific questions:

why do we need to determine the distribution of 𝐘?

We don't. We make assumptions about the conditional distribution of $\bf{Y}$, not the marginal.

how do we interpret each component of 𝐘=(π‘Œ1,π‘Œ2,…,π‘Œπ‘›)?

As a draw from $$y \vert \beta,x \sim \mathcal{N}(x^T\beta, \sigma)$$

Does each π‘Œπ‘– represent the outcome from a different sample?


| cite | improve this answer | |
  • $\begingroup$ The problem which concerns me is the distribution $\textbf{Y}\sim\mathcal{N}(\textbf{X}\beta,\sigma^{2}\textbf{I})$. How do we interpret it? According to such distribution, each component $Y_{i}$ could have a different mean. $\endgroup$ – user1337 May 16 '19 at 14:11
  • $\begingroup$ @user1337 That is correct. Each element of the response vector should come from a normal with distinct mean, namely $x^T\beta$. $\endgroup$ – Demetri Pananos May 16 '19 at 14:16
  • $\begingroup$ Sorry for insisting, but I still did not understand why they have different means. In simple linear regression, we colect the pairs of data $(x_{i},y_{i})$ and estimate the parameter $\beta$. In this case, all the information is collected from a single sample, which has an unique sample mean. When we extend the study to the multiple linear regression case, why do we get different means? Don't the data come from the same sample? $\endgroup$ – user1337 May 16 '19 at 14:20
  • 1
    $\begingroup$ I think you are hung up on the difference between conditional and marginal means. Focus on a single element of $\bf{Y}$. It has associated covariates $x$. I were to sample the phenomenon again and again, linear regression assumes that those samples would be normally distributed with mean $x^T\beta$. Linear regression doesn't care about the distribution or sample mean of $\bf{Y}$ $\endgroup$ – Demetri Pananos May 16 '19 at 14:26
  • $\begingroup$ I think I get it now. If we are given $n$ observations $Y_{i}$ and the model matrix $\textbf{X}$, we determine $\hat{\beta}$, where $x^{T}\hat{\beta}$ is an estimator for the mean $\mu$. Such procedure results into the distribution $\mathcal{N}(x^{T}\hat{\beta},\sigma^{2})$ for $y$. Am I on the right track? Anyway, there still remains one question: does it have any purpose on obtaining the distribution of $\textbf{Y}$? $\endgroup$ – user1337 May 16 '19 at 14:38

Your Answer

By clicking β€œPost Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.