4
$\begingroup$

Once again, here I am. Given the multiple linear regression model \begin{align*} \textbf{Y} = \textbf{X}\beta + \epsilon \end{align*}

where $\epsilon\sim\mathcal{N}(\textbf{0},\sigma^{2}\textbf{I})$ and $\mu = \textbf{X}\beta$, why do we need to determine the distribution of $\textbf{Y}$? If we apply the least square method to obtain $\hat{\beta}$, we get the explicit relation \begin{align*} Y_{i} = \hat{\beta}_{0} + \hat{\beta}_{1}x_{i1} + \ldots + \hat{\beta}_{p-1}x_{i,p-1} + \epsilon_{i} \end{align*}

from whence we are able to obtain the value of the response variable $Y$ in terms of the explanatory variables. My second question is: how do we interpret each component of $\textbf{Y} = (Y_{1},Y_{2},\ldots,Y_{n})$? Does each $Y_{i}$ represent the outcome from a different sample? Otherwise, if they belong to the same sample, why do they have different means?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Linear regression makes no assumptions on the distribution of the marginal outcome (That is, $\bf{Y}$). However, there is an assumption on the distribution of the elements of $\bf{Y}$.

Each element of $\bf{Y}$ should have a normal distribution. Given the covariates, the distribution should be

$$y \vert \beta,x \sim \mathcal{N}(x^T\beta, \sigma)$$

To your specific questions:

why do we need to determine the distribution of 𝐘?

We don't. We make assumptions about the conditional distribution of $\bf{Y}$, not the marginal.

how do we interpret each component of 𝐘=(𝑌1,𝑌2,…,𝑌𝑛)?

As a draw from $$y \vert \beta,x \sim \mathcal{N}(x^T\beta, \sigma)$$

Does each 𝑌𝑖 represent the outcome from a different sample?

Yes

Improve this answer
$\endgroup$
11
  • $\begingroup$ The problem which concerns me is the distribution $\textbf{Y}\sim\mathcal{N}(\textbf{X}\beta,\sigma^{2}\textbf{I})$. How do we interpret it? According to such distribution, each component $Y_{i}$ could have a different mean. $\endgroup$
    – user242554
    Commented May 16, 2019 at 14:11
  • $\begingroup$ @user1337 That is correct. Each element of the response vector should come from a normal with distinct mean, namely $x^T\beta$. $\endgroup$
    – Demetri Pananos
    Commented May 16, 2019 at 14:16
  • $\begingroup$ Sorry for insisting, but I still did not understand why they have different means. In simple linear regression, we colect the pairs of data $(x_{i},y_{i})$ and estimate the parameter $\beta$. In this case, all the information is collected from a single sample, which has an unique sample mean. When we extend the study to the multiple linear regression case, why do we get different means? Don't the data come from the same sample? $\endgroup$
    – user242554
    Commented May 16, 2019 at 14:20
  • 2
    $\begingroup$ I think you are hung up on the difference between conditional and marginal means. Focus on a single element of $\bf{Y}$. It has associated covariates $x$. I were to sample the phenomenon again and again, linear regression assumes that those samples would be normally distributed with mean $x^T\beta$. Linear regression doesn't care about the distribution or sample mean of $\bf{Y}$ $\endgroup$
    – Demetri Pananos
    Commented May 16, 2019 at 14:26
  • $\begingroup$ I think I get it now. If we are given $n$ observations $Y_{i}$ and the model matrix $\textbf{X}$, we determine $\hat{\beta}$, where $x^{T}\hat{\beta}$ is an estimator for the mean $\mu$. Such procedure results into the distribution $\mathcal{N}(x^{T}\hat{\beta},\sigma^{2})$ for $y$. Am I on the right track? Anyway, there still remains one question: does it have any purpose on obtaining the distribution of $\textbf{Y}$? $\endgroup$
    – user242554
    Commented May 16, 2019 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.