2
$\begingroup$

Does it matter if a dependent variable has a normal distribution or not when it is used in a regression?

Improve this question
$\endgroup$
2

2 Answers 2

Reset to default
5
$\begingroup$

Clarification: the normality assumption relates to the conditional distribution of the response (the DV), not its marginal distribution. The shape of the distribution of the DV by itself may be quite non-normal, depending on the arrangement of predictors (IVs). The assumption about the conditional distribution is equivalent to an assumption of normal errors in the model.

i) To fit a regression by least squares doesn't require any assumption of normality

ii) Inference on a least squares regression is commonly based on a normal assumption

iii) In large samples, many of the properties of the usual tests and confidence intervals are not that sensitive to normality, though the extent does depend on how non-normal the distribution is, as well as on the pattern of $X$'s. More precisely, efficiency-related considerations like power will be impacted however, but in large samples type I error would not usually be a major concern.

iv) The usual prediction intervals are, however, relatively more sensitive to the normality assumption, since they deal with distributions of single values.

v) There are a variety of alternative forms of both fitting and inference that could be applied to linear fitting under non-normality.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Just to be super explicit: (in my view) Glen_b is not talking about the normality of the dependent variable, but the normality of the errors in the Y dimension about the line of means. $\endgroup$
    – Alexis
    Commented Jun 30, 2014 at 16:18
  • $\begingroup$ @Alexis Yes, that's correct, but your answer already contained that information, so I didn't repeat it. I'll put a sentence in to be more explicit. $\endgroup$
    – Glen_b
    Commented Jun 30, 2014 at 21:43
  • $\begingroup$ @Glen_b For (ii), inference you mean hypothesis testing? Or inference is referred to (iii) and (iv)? $\endgroup$
    – Vickyyy
    Commented Nov 14, 2018 at 17:19
  • $\begingroup$ By inference I meant confidence intervals, predictions intervals and hypothesis testing (though there are a few additional things that might come under inference); these infer properties of unknown / unobserved / unobservable quantities (for tests and CIs, its some population parameter or some combination of them) - and do so under some set of assumptions about that population. $\endgroup$
    – Glen_b
    Commented Nov 14, 2018 at 23:03
4
$\begingroup$

Nope.

What matters is the normality of the errors.

Here's a simple exercise to convince yourself that this is so:

  • Assume that the true model is $y = 3 + 0.5x + \varepsilon$, where $\varepsilon \sim \mathcal{n}(0,1)$, and $x \sim \mathcal{U}(0,100)$

  • Model this relationship using OLS as $y_{i} = \beta_{0} + \beta_{x}x_{i} +\varepsilon_{i}$, where $\varepsilon_{i} \sim \mathcal{n}(0,\sigma)$ to obtain the estimates $\hat{\beta}_{0}$, $\hat{\beta}_{x}$, $\hat{\varepsilon}_{i}$, and $\hat{\sigma}$.

  • Then simulate data from this model across, say, $N=100$ observations in R:

    x <- 100*runif(100)
y <- 3 + 0.5*x + rnorm(100,0,1)

    Applying the regression model above: 

    summary(lm(y~x))

    you will obtain $\hat{\beta}_{0} \approx 3$ and $\hat{\beta}_{x} \approx 0.5$. The kicker is in the decidedly non-normal histograms of $y$ and $x$:

    hist(x)
hist(y)

    And of course, the distribution of $\hat{\varepsilon}_{i}$ will be about normal:

    e <- y - 3 + 0.5*x
hist(e)

The distributions of $y$ and $x$, while assumed i.i.d., have no assumed distribution.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.