5
$\begingroup$

In linear regression, if the random error is normally distributed, does this mean the response is normal as well? In particular if $\epsilon$ ~ N(0,$\sigma^2$) does this mean Y~N($\alpha + \beta X$, $\sigma^2$). More specifically, I am asking if Y will have a normal distribution. I know the mean will be $\alpha + \beta X$ and variance will be $\sigma^2$ but can the distribution be assumed to be normal just because $\epsilon$ is normal? Why?

$\endgroup$
  • 2
    $\begingroup$ Possible duplicate of Where does the misconception that Y must be normally distributed come from? $\endgroup$ – Ben Nov 7 '18 at 23:22
  • 1
    $\begingroup$ @Ben that link there seems more like "what are all these different sets of assumptions?" while this question seems like "why are affine transformations of normal rvs still normal?" $\endgroup$ – Taylor Nov 7 '18 at 23:30
  • $\begingroup$ @s5s you can show this with moment generating functions. Have you tried that? $\endgroup$ – Taylor Nov 7 '18 at 23:32
  • 1
    $\begingroup$ It's important to distinguish between the conditional distribution of Y given the value of X and the marginal distribution of Y. $\endgroup$ – Glen_b Nov 8 '18 at 0:41
4
$\begingroup$

As is pointed out in this related question, the normality of the error term in a linear regression is not sufficient to ensure the marginal normality of the response variable. The latter is also affected by the distribution of the explanatory variable, which is not assumed to be normal in a regression analysis.

Under the linear regression model you have specified, the conditional distribution of $Y$ is:

$$Y|x \sim \text{N}(\alpha + \beta x, \sigma^2).$$

The marginal distribution of $Y$ is:

$$F_Y(y) \equiv \mathbb{P}(Y \leqslant y) = \int \limits_{-\infty}^\infty \Phi \Big( \frac{y - \alpha - \beta x}{\sigma} \Big) f_X(x) dx,$$

where $\Phi$ is the CDF of the standard normal distribution. In the special case where $X \sim \text{N}$ this leads to a normal distribution, but in the more general case where the explanatory variable has some other distribution, you will often get a marginal distribution for the response variable that is not normal.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ The problem with your answer about the conditional distribution of $Y$ is that you are assuming that the error terms have conditionally normal distributions, but this is not the usual implication. The marginal distribution of the errors is what is used to check normality of the conditional $Y$ distributions. And marginal normality of $\epsilon$ certainly does not imply conditional normality of the $Y$ distributions; see stats.stackexchange.com/a/486951/102879 $\endgroup$ – BigBendRegion Sep 13 at 12:51
  • $\begingroup$ All regression results are conditional on $x$, so it is usual to assume conditional normality for the errors. $\endgroup$ – Ben Sep 13 at 22:35
  • $\begingroup$ But that is not how the diagnostic procedure is used - it is based on the aggregate residuals, not on the residuals within each level of $X$. $\endgroup$ – BigBendRegion Sep 13 at 23:04
  • $\begingroup$ I'm not sure I follow you. The residuals are functions of both the response and the explanatory variables, so they already account for variation in the latter. The usual diagnostic is to plot the studentised residuals against the predicted values. $\endgroup$ – Ben Sep 13 at 23:30
  • $\begingroup$ That plot somewhat conditions on the $x$ data, but not completely with multiple regression. But that plot is not usually intended to diagnose normality. Rather, the residual histogram and q-q plots are the specific diagnostic tools for the regression normality assumption. These tools clearly investigate the marginal distribution of the residuals. And as the example in stats.stackexchange.com/a/486951/102879 shows, such diagnostics can show normality (and correspond to theoretical normality) when the conditional $Y$ distributions are non-normal (and even homoscedastic). $\endgroup$ – BigBendRegion Sep 14 at 14:02
2
$\begingroup$

The answer is a most definitive "no." Marginal normality of $\epsilon$ does not imply that the conditional distributions of $Y$ are normal. See here for a counterexample:

https://stats.stackexchange.com/a/486951/102879

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

The distribution at a fixed value of x is normal. Y is not normal. Just look at the histogram of the response. It will not look like a normal distribution. But if you look at the distribution at a fixed x, then it will look normal.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

Yes if $\varepsilon \sim N(0, \sigma^2)$ $Y=\alpha+\beta x+ \epsilon$ then we can say that $Y \sim N(\alpha+\beta x,\sigma^2)$. This follows from the result that if a random variable $X \sim N(\mu, \sigma^2)$ then $X+a \sim N(\mu+a, \sigma^2)$, for example if $X\sim N(0, 3^2)$ then $X+2 \sim N(2, 3^2)$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ The fact that if a random variable $X\sim N(\mu,\sigma^2)$ then $X+a∼N(\mu+a,\sigma^2)$ can be established using the method of moment generating functions. $\endgroup$ – user22546 Nov 7 '18 at 23:48
  • $\begingroup$ What is missing here is an implicit assumption that $\epsilon$ is normally distributed conditional on X. But the usual diagnostics are done unconditionally, so this is not a relevant assumption. And further, unconditional normality of the $\epsilon$ does not imply conditional normality of $Y$. See stats.stackexchange.com/a/486951/102879 $\endgroup$ – BigBendRegion Sep 13 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.