6
$\begingroup$

In linear regression, if the random error is normally distributed, does this mean the response is normal as well? In particular if $\epsilon$ ~ N(0,$\sigma^2$) does this mean Y~N($\alpha + \beta X$, $\sigma^2$). More specifically, I am asking if Y will have a normal distribution. I know the mean will be $\alpha + \beta X$ and variance will be $\sigma^2$ but can the distribution be assumed to be normal just because $\epsilon$ is normal? Why?

$\endgroup$
4
  • 2
    $\begingroup$ Possible duplicate of Where does the misconception that Y must be normally distributed come from? $\endgroup$
    – Ben
    Commented Nov 7, 2018 at 23:22
  • 1
    $\begingroup$ @Ben that link there seems more like "what are all these different sets of assumptions?" while this question seems like "why are affine transformations of normal rvs still normal?" $\endgroup$
    – Taylor
    Commented Nov 7, 2018 at 23:30
  • $\begingroup$ @s5s you can show this with moment generating functions. Have you tried that? $\endgroup$
    – Taylor
    Commented Nov 7, 2018 at 23:32
  • 1
    $\begingroup$ It's important to distinguish between the conditional distribution of Y given the value of X and the marginal distribution of Y. $\endgroup$
    – Glen_b
    Commented Nov 8, 2018 at 0:41

4 Answers 4

4
$\begingroup$

As is pointed out in this related question, the normality of the error term in a linear regression is not sufficient to ensure the marginal normality of the response variable. The latter is also affected by the distribution of the explanatory variable, which is not assumed to be normal in a regression analysis.

Under the linear regression model you have specified, the conditional distribution of $Y$ is:

$$Y|x \sim \text{N}(\alpha + \beta x, \sigma^2).$$

The marginal distribution of $Y$ is:

$$F_Y(y) \equiv \mathbb{P}(Y \leqslant y) = \int \limits_{-\infty}^\infty \Phi \Big( \frac{y - \alpha - \beta x}{\sigma} \Big) f_X(x) dx,$$

where $\Phi$ is the CDF of the standard normal distribution. In the special case where $X \sim \text{N}$ this leads to a normal distribution, but in the more general case where the explanatory variable has some other distribution, you will often get a marginal distribution for the response variable that is not normal.

$\endgroup$
7
  • $\begingroup$ The problem with your answer about the conditional distribution of $Y$ is that you are assuming that the error terms have conditionally normal distributions, but this is not the usual implication. The marginal distribution of the errors is what is used to check normality of the conditional $Y$ distributions. And marginal normality of $\epsilon$ certainly does not imply conditional normality of the $Y$ distributions; see stats.stackexchange.com/a/486951/102879 $\endgroup$ Commented Sep 13, 2020 at 12:51
  • $\begingroup$ All regression results are conditional on $x$, so it is usual to assume conditional normality for the errors. $\endgroup$
    – Ben
    Commented Sep 13, 2020 at 22:35
  • $\begingroup$ But that is not how the diagnostic procedure is used - it is based on the aggregate residuals, not on the residuals within each level of $X$. $\endgroup$ Commented Sep 13, 2020 at 23:04
  • $\begingroup$ I'm not sure I follow you. The residuals are functions of both the response and the explanatory variables, so they already account for variation in the latter. The usual diagnostic is to plot the studentised residuals against the predicted values. $\endgroup$
    – Ben
    Commented Sep 13, 2020 at 23:30
  • $\begingroup$ That plot somewhat conditions on the $x$ data, but not completely with multiple regression. But that plot is not usually intended to diagnose normality. Rather, the residual histogram and q-q plots are the specific diagnostic tools for the regression normality assumption. These tools clearly investigate the marginal distribution of the residuals. And as the example in stats.stackexchange.com/a/486951/102879 shows, such diagnostics can show normality (and correspond to theoretical normality) when the conditional $Y$ distributions are non-normal (and even homoscedastic). $\endgroup$ Commented Sep 14, 2020 at 14:02
3
$\begingroup$

The answer is a most definitive "no." Marginal normality of $\epsilon$ does not imply that the conditional distributions of $Y$ are normal. See here for a counterexample:

https://stats.stackexchange.com/a/486951/102879

$\endgroup$
1
$\begingroup$

The distribution at a fixed value of x is normal. Y is not normal. Just look at the histogram of the response. It will not look like a normal distribution. But if you look at the distribution at a fixed x, then it will look normal.

$\endgroup$
0
$\begingroup$

Yes if $\varepsilon \sim N(0, \sigma^2)$ $Y=\alpha+\beta x+ \epsilon$ then we can say that $Y \sim N(\alpha+\beta x,\sigma^2)$. This follows from the result that if a random variable $X \sim N(\mu, \sigma^2)$ then $X+a \sim N(\mu+a, \sigma^2)$, for example if $X\sim N(0, 3^2)$ then $X+2 \sim N(2, 3^2)$

$\endgroup$
2
  • $\begingroup$ The fact that if a random variable $X\sim N(\mu,\sigma^2)$ then $X+a∼N(\mu+a,\sigma^2)$ can be established using the method of moment generating functions. $\endgroup$
    – user22546
    Commented Nov 7, 2018 at 23:48
  • $\begingroup$ What is missing here is an implicit assumption that $\epsilon$ is normally distributed conditional on X. But the usual diagnostics are done unconditionally, so this is not a relevant assumption. And further, unconditional normality of the $\epsilon$ does not imply conditional normality of $Y$. See stats.stackexchange.com/a/486951/102879 $\endgroup$ Commented Sep 13, 2020 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.