2
$\begingroup$

Does it matter if a dependent variable has a normal distribution or not when it is used in a regression?

$\endgroup$
4
$\begingroup$

Clarification: the normality assumption relates to the conditional distribution of the response (the DV), not its marginal distribution. The shape of the distribution of the DV by itself may be quite non-normal, depending on the arrangement of predictors (IVs). The assumption about the conditional distribution is equivalent to an assumption of normal errors in the model.

i) To fit a regression by least squares doesn't require any assumption of normality

ii) Inference on a least squares regression is commonly based on a normal assumption

iii) In large samples, many of the properties of the usual tests and confidence intervals are not that sensitive to normality, though the extent does depend on how non-normal the distribution is, as well as on the pattern of $X$'s. More precisely, efficiency-related considerations like power will be impacted however, but in large samples type I error would not usually be a major concern.

iv) The usual prediction intervals are, however, relatively more sensitive to the normality assumption, since they deal with distributions of single values.

v) There are a variety of alternative forms of both fitting and inference that could be applied to linear fitting under non-normality.

$\endgroup$
  • 1
    $\begingroup$ Just to be super explicit: (in my view) Glen_b is not talking about the normality of the dependent variable, but the normality of the errors in the Y dimension about the line of means. $\endgroup$ – Alexis Jun 30 '14 at 16:18
  • $\begingroup$ @Alexis Yes, that's correct, but your answer already contained that information, so I didn't repeat it. I'll put a sentence in to be more explicit. $\endgroup$ – Glen_b -Reinstate Monica Jun 30 '14 at 21:43
  • $\begingroup$ @Glen_b For (ii), inference you mean hypothesis testing? Or inference is referred to (iii) and (iv)? $\endgroup$ – Vickyyy Nov 14 '18 at 17:19
  • $\begingroup$ By inference I meant confidence intervals, predictions intervals and hypothesis testing (though there are a few additional things that might come under inference); these infer properties of unknown / unobserved / unobservable quantities (for tests and CIs, its some population parameter or some combination of them) - and do so under some set of assumptions about that population. $\endgroup$ – Glen_b -Reinstate Monica Nov 14 '18 at 23:03
4
$\begingroup$

Nope.

What matters is the normality of the errors.

Here's a simple exercise to convince yourself that this is so:

  • Assume that the true model is $y = 3 + 0.5x + \varepsilon$, where $\varepsilon \sim \mathcal{n}(0,1)$, and $x \sim \mathcal{U}(0,100)$

  • Model this relationship using OLS as $y_{i} = \beta_{0} + \beta_{x}x_{i} +\varepsilon_{i}$, where $\varepsilon_{i} \sim \mathcal{n}(0,\sigma)$ to obtain the estimates $\hat{\beta}_{0}$, $\hat{\beta}_{x}$, $\hat{\varepsilon}_{i}$, and $\hat{\sigma}$.

  • Then simulate data from this model across, say, $N=100$ observations in R:

    x <- 100*runif(100)
    y <- 3 + 0.5*x + rnorm(100,0,1)
    

    Applying the regression model above:

    summary(lm(y~x))
    

    you will obtain $\hat{\beta}_{0} \approx 3$ and $\hat{\beta}_{x} \approx 0.5$. The kicker is in the decidedly non-normal histograms of $y$ and $x$:

    hist(x)
    hist(y)
    

    And of course, the distribution of $\hat{\varepsilon}_{i}$ will be about normal:

    e <- y - 3 + 0.5*x
    hist(e)
    

The distributions of $y$ and $x$, while assumed i.i.d., have no assumed distribution.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.